Databricks Platform Discussions
Dive into comprehensive discussions covering various aspects of the Databricks platform. Join the co...
11268 Posts
Activity in Discussions
- 14 Views
- 1 replies
- 0 kudos
Specify cluster when using dbutils.notebook API
Hello! Anyone know if there is a way of specifying cluster when using dbutils.notebook.run() ? As i understand it, this command will create a job compute on the run? but what if I want to use my general purpose cluster? I have been thinking of %run b...
- 14 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @ErikJ ,When you run a child notebook from parent notebook, the child notebook will use the same cluster.
- 0 kudos
- 1450 Views
- 4 replies
- 1 kudos
Resolved! Hard reset programatically
Is it possible to trigger a git reset --hard programatically?I'm running a platform service where, as part of CI/CD, repos get deployed into the Databricks workspace. Normally, our developers work with upstream repos both from their local IDEs and fr...
- 1450 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
Hi from the Git folders/Repos PM: We don't currently have a solution to hard reset programmatically. This is on our roadmap. If you orchestrate production jobs using the Databricks workflows, there is an option to use version-controlled source code i...
- 1 kudos
- 31 Views
- 1 replies
- 0 kudos
Delta Table Log History not Updating
I am running into an issue related to my Delta Log and an old version. I currently have default delta settings for delta.checkpointInterval (10 commits as this table was created prior to DBR 11.1), delta.deletedFileRetentionDuration (7 days), and del...
- 31 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
@Dave_Nithio wrote:I am running into an issue related to my Delta Log and an old version. I currently have default delta settings for delta.checkpointInterval (10 commits as this table was created prior to DBR 11.1), delta.deletedFileRetentionDuratio...
- 0 kudos
- 11 Views
- 0 replies
- 0 kudos
Connection Refused: [Databricks][JDBC](11640) Required Connection Key(s): PWD;
Hey I'm trying to connect to Databricks using client id and secrets. I'm using JDBC 2.6.38.I'm using the following connection url: jdbc:databricks://<server-hostname>:443;httpPath=<http-path>;AuthMech=11;Auth_Flow=1;OAuth2ClientId=<service-principal-...
- 11 Views
- 0 replies
- 0 kudos
- 77 Views
- 2 replies
- 0 kudos
Lost untracked and gitignore files when push new commit
Hi guys, our team uses Databricks clusters to develop with Python using Jupyter Notebooks.Recently we had a serious problem in our repos folder ( mainly Jupyter Notebook and Python scripts ).After my teammate committed in the branch, all the tracked ...
- 77 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Yes we have a subscription. Please tell me which information you need to check. I want to dive in this problem. If possible, pls contact through my email.
- 0 kudos
- 20 Views
- 0 replies
- 0 kudos
How to fully enable row-level concurrency on Databricks 14.1
Hey guys,I hope whoever's reading this is doing well.We're trying to enable row-level concurrency on Databricks 14.1. However, the documentation seems a bit contradictory as to whether this is possible, and whether the whole capability of row-level c...
- 20 Views
- 0 replies
- 0 kudos
- 83 Views
- 1 replies
- 0 kudos
" ResourceNotFound" error is coming on connecting devops repo to databricks workflow(job).
I have a .py file in a repo in azure devops,I want to add it in a workflow in databricks and these are the values I have provided. And the source is this:I have provided all the values correctly but getting this error: " ResourceNotFound". Can someon...
- 83 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Can you try cloning the DevOps repo as a Git folder? The git folder clone interface should ask you to set up a Git credential if it's not already there.
- 0 kudos
- 5159 Views
- 3 replies
- 3 kudos
Resolved! Import from repo
Hi all,I am trying the new "git folder" feature, with a repo that works fine from the "Repos". In the new folder location, my imports from my own repo don't work anymore. Anyone faced something similar?Thanks in advance for sharing your experience
- 5159 Views
- 3 replies
- 3 kudos
Latest Reply
- 3 kudos
Hi from the Git folder product manager - If you use DBR 14.3+, the Git folder root is automatically added to the Python `sys.path`. This is documented here. Unfortunately we could not backport this behavior to earlier DBR versions. Hope this helps!...
- 3 kudos
- 2037 Views
- 4 replies
- 0 kudos
DBUtils commands do not work on shared access mode clusters
Hi there,I am trying to upload a file to an s3 bucket. However, none of dbutils commands seem to work neither does the boto3 library. For clusters that have the configuration, except for the shared access mode, seem to work fine.Those are the error m...
- 2037 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
I am encountering very similar behavior to drii_cavalcanti. When I use a Shared cluster with an IAM Role specified I can verify that the aws cli is installed but when I run aws sts get-caller-identity I receive the error "Unable to locate credential...
- 0 kudos
- 28 Views
- 0 replies
- 0 kudos
luv scrub amazon
Exfoliating Washcloth Face & Body Scrub Towel — Japanese Exfoliating Towel with Hexagon Fibers, Exfoliating Body Scrubber with 2 Sides for Scrubbing & Washing — 1 Extra Long TowelReveals Softer & Smoother Skin — Get silky soft skin that glows with ou...
- 28 Views
- 0 replies
- 0 kudos
- 40 Views
- 0 replies
- 0 kudos
Resource organization in a large company
Hello,We are using Azure Databricks in a single tenant. We will have many teams working in multiple (Unity Enabled) Workspaces using a variety of Catalogs, External Locations, Storage Credentials, ect. Some of those resources will be shared (e.g., an...
- 40 Views
- 0 replies
- 0 kudos
- 202 Views
- 3 replies
- 0 kudos
How to stop a continuous pipeline which is set to RETRY on FAILURE and failing for some reason
I have created a pipeline which is continuous and set to RETRY on FAILURE. For some reason it keeps failing and retrying. Is there any way I can stop it. Hitting Stop button throws an error.
- 202 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @szymon_dybczak I already tried to remove it via REST API but got same error as in the pipeline logs. Eventually, I had to remove workspace to get rid of it.
- 0 kudos
- 46 Views
- 4 replies
- 1 kudos
How to get catalog and schema from system query table
Hi,We are querying the system.query table to parse query history. If the table in the query is not fully qualified with its catalog and schema, how can we derive the catalog and schema?Thanks,Jen
- 46 Views
- 4 replies
- 1 kudos
Latest Reply
- 1 kudos
There is no straightforward method to get this data. Run the query to check the defaults:SELECT current_catalog() AS default_catalog, current_schema() AS default_schema;Catalog and schema may be changed in the query, so if you have query text you...
- 1 kudos
- 28 Views
- 0 replies
- 0 kudos
Bring data from databricks to sharepoint list using the Power Automate
Good afternoon to all and I am new to this community.We are trying to bring data from databricks to sharepoint list using the Power Automate app (create workflow and trigger it when there is new record or exising record is modified in source table in...
- 28 Views
- 0 replies
- 0 kudos
- 2129 Views
- 4 replies
- 2 kudos
Editor lags
Does anyone else experiencing lags on the Databricks notebook editor?The typing is sometimes very slow causes the cell to hang for a moment. (Using python)
- 2129 Views
- 4 replies
- 2 kudos
Latest Reply
- 2 kudos
The notebook editor has been sluggish for me regardless of splitting up between cells and the language in use. Very frustrating.
- 2 kudos
| User | Count |
|---|---|
| 1811 | |
| 861 | |
| 473 | |
| 312 | |
| 308 |
