I'm just learning to use mosaic and delta live table in Databricks. I was following this example https://github.com/databrickslabs/mosaic/tree/main/notebooks/examples/python/OpenStreetMaps and after configuring everything I'm not able to run the pipe...
Hi,I would like to start using Delta Sharing but I would need to check if my tables hit limitations which are in the following article: RESOURCE_LIMIT_EXCEEDED error when querying a Delta Sharing table - Databricks.Delta Sharing has limits on the met...
Hi All,We are facing an issue while creating a simple job of type "python script". A python file in workspace is selected as a source. No arguments/job parameters are provided. This is a strange behavior and just started occurring since today morning...
Hi Ayushi,How can I call call RESET API? this issue is occurring when creating a new job from databricks web UI. It looks like REST API is for resetting job settings of an existing job.Can this be an issue with the databricks workspace we are using?A...
We have an Azure app service written in Django. From databricks notebook we sent curl command to test the connection between databricks and Azure AppService. We got the following repsonse:Error 403 - ForbiddenThe web app you have attempted to reach...
Hello, I have a question.Context :I have a Unity Catalog organized with three schemas (bronze, silver and gold). Logically, I would like to create tables in each schemas.I tried to organize my pipelines on the layers, which mean that I would like to ...
Hello, thanks for the answers @YuliyanBogdanov, @standup1.So the solution is to use catalog.schema.table, and not LIVE.table, that's the key, you were right standup!But, you won't have the visibility of the tables on Bronze Pipeline, if you are on Si...
I have a source table A in Unity Catalog. This table is constantly written to and is a streaming table.I also have another table B in Unity Catalog. This is a managed table with liquid clustering.Using Auto Loader I move new data from A to B using a ...
change data feed might be a solution for you perhaps.https://docs.databricks.com/en/delta/delta-change-data-feed.html
Could someone please help me with the below question:1. How much time does the VM takes to deallocate the CPUs?2. Can we check how much quota is required to run a script?
Hi,We are trying to use Python SDK and create a workspace client using the following code:%pip install databricks-sdk --upgrade dbutils.library.restartPython()from databricks.sdk import WorkspaceClientw = WorkspaceClient()Here is the notebook: https:...
Hi @databrick_usert , Hope you are doing well! Can you check the version of the SDK running in this notebook? If it's not an upgraded version then could you please try to upgrade the SDK version and then restart the python after the pip install? %p...
Looking for a solution to delete 1000's of non-empty folders that I created wrongly while migration. (looking for solution that delete thoudands of folders created with below name)Folders have a path like "shared/db/..../..../..../yippy.dbc"
Hi I have a requirement to read table from azure sql db and update the table in azure databricks with transformations and overwrite updated table to the azure sql db but due to lazy evaluation of pyspark im unable to overwrite the table in azure sql ...
Hi @Kingston Make sure that you have the proper permissions on the SQL server for the user you do the authentication through JDBC with, i.e. database reader / database writer. Then your approach can go in two directions, push the data from Databrick...
I discovered recently mlflow managed by Databricks so I'm very new to this and I need some help.Can someone explain for me clearly the steps to do to be able to track my runs into the Databricks API.Here are the steps I followed :1/ Installing Databr...
I am unable to write data from Databricks into an S3 bucket. I have set up the permissions both on the bucket policy level, and the user level as well (Put, List, and others are added, have also tried with s3*). Bucket region and workspace region are...
Hello,For the purposes of testing I'm interested in creating a new workspace with Unity Catalog enabled, and from there I'd like to access (external - S3) tables on an existing legacy hive metastore workspace (not UC enabled). The goal is for both wo...
Your aim is to access external S3 tables from a Unity Catalog workspace without data duplication and keeping data updates synchronized. Configure external location permissions. This ensure that both your Unity Catalog and Hive metastore workspaces h...
I have a databricks function that returns a table_nameCREATE OR REPLACE FUNCTION test_func() RETURNS string READS SQL DATA RETURN 'table_name'I want to select from the table that is returned by this function. How can I make it work in SQL, some...
I looked at this posthttps://stackoverflow.com/questions/77475436/in-databricks-workbook-using-spark-sql-how-to-pass-parameters-thru-sql-udf-func What I want is to replace the static table name with the table name passed as parameter (param_table_nam...
Trying to start a new compute this morning and I get the "Self Bootstrap Error." I saw some people with similar error but their fixes involved DNS settings. I've haven't made any dns changes and was working fine without any problem last night. I real...
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 486 | |
| 349 | |
| 287 |
