I am following this guide on allowing OAuth U2M for Azure Databricks.When I get to Step 2, I make a request to account.azuredatabricks.net and specify a redirect URI to receive a code.The redirect URI in the example is localhost:8020. If I change thi...
Databricks Repos best-practices recommend using the Repos REST API to update a repo via your git provider. The REST API requires authentication, which can be done one of two ways:A user / personal access tokenA service principal access tokenUsing a u...
Having the exact same problem. Did you find a solution @michael_mehrten ?In my case Im using a managed identity so the solution some topics suggest on generating an access token from a Entra ID service principal is not applicable.
I'm encountering some issues while trying to read a public dataset from a URL using Databricks. Here's the code snippet(along with errors) I'm working with: I'm confused about Delta format error here.When I read data from a URL, how would it have a D...
@sharma_kamal Please disable the formatCheck in notebook and check if you could read the data The configuration command %sql SET spark.databricks.delta.formatCheck.enabled=false will disable the format check for Delta tables in Databricks. Databrick...
Hi everyone,I'm trying to use Git provider in Databricks job.First, I was using my personal user account to `Run as`.But when I change `Run as` to Service Principal, it was failed because of permission error.And I can't find a way to solve it.Could I...
The documentation is lacking in this area which should be easy to set up. Instead we are forced to search among community topics such as these.
Hi guys, I've encountered an issue while attempting to query shared views via Delta Sharing in Databricks. We are using Delta Sharing Databricks-to-Databricks protocol to share data from a databricks environment deployed on azure to another databrick...
I'm just learning to use mosaic and delta live table in Databricks. I was following this example https://github.com/databrickslabs/mosaic/tree/main/notebooks/examples/python/OpenStreetMaps and after configuring everything I'm not able to run the pipe...
Hi,I would like to start using Delta Sharing but I would need to check if my tables hit limitations which are in the following article: RESOURCE_LIMIT_EXCEEDED error when querying a Delta Sharing table - Databricks.Delta Sharing has limits on the met...
Hi All,We are facing an issue while creating a simple job of type "python script". A python file in workspace is selected as a source. No arguments/job parameters are provided. This is a strange behavior and just started occurring since today morning...
Hi Ayushi,How can I call call RESET API? this issue is occurring when creating a new job from databricks web UI. It looks like REST API is for resetting job settings of an existing job.Can this be an issue with the databricks workspace we are using?A...
We have an Azure app service written in Django. From databricks notebook we sent curl command to test the connection between databricks and Azure AppService. We got the following repsonse:Error 403 - ForbiddenThe web app you have attempted to reach...
Hi,We have been planning to migrate the Synapse Data bricks activity executions from 'All-purpose cluster' to 'New job cluster' to reduce overall cost. We are using Standard_D3_v2 as cluster node type that has 4 CPU cores in total. The current quota ...
I also see a difference in Photon, Enable Photon for workloads with large data scans, joins, aggregations, and decimal computations. Photon provides significant performance benefits over the standard Databricks Runtime.
Hello, I have a question.Context :I have a Unity Catalog organized with three schemas (bronze, silver and gold). Logically, I would like to create tables in each schemas.I tried to organize my pipelines on the layers, which mean that I would like to ...
Hello, thanks for the answers @YuliyanBogdanov, @standup1.So the solution is to use catalog.schema.table, and not LIVE.table, that's the key, you were right standup!But, you won't have the visibility of the tables on Bronze Pipeline, if you are on Si...
I have a source table A in Unity Catalog. This table is constantly written to and is a streaming table.I also have another table B in Unity Catalog. This is a managed table with liquid clustering.Using Auto Loader I move new data from A to B using a ...
change data feed might be a solution for you perhaps.https://docs.databricks.com/en/delta/delta-change-data-feed.html
Could someone please help me with the below question:1. How much time does the VM takes to deallocate the CPUs?2. Can we check how much quota is required to run a script?
Hi,We are trying to use Python SDK and create a workspace client using the following code:%pip install databricks-sdk --upgrade dbutils.library.restartPython()from databricks.sdk import WorkspaceClientw = WorkspaceClient()Here is the notebook: https:...
Hi @databrick_usert , Hope you are doing well! Can you check the version of the SDK running in this notebook? If it's not an upgraded version then could you please try to upgrade the SDK version and then restart the python after the pip install? %p...
Looking for a solution to delete 1000's of non-empty folders that I created wrongly while migration. (looking for solution that delete thoudands of folders created with below name)Folders have a path like "shared/db/..../..../..../yippy.dbc"
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group| User | Count |
|---|---|
| 1610 | |
| 763 | |
| 345 | |
| 286 | |
| 251 |
