- 4568 Views
- 3 replies
- 3 kudos
Set Not null changes Data type
Hello,Just found this issue this week and thought I would ask. An Alter Table alter column set not null is changing a varchar(x) data type to string type. I believe this should happen in most environments so I wouldn't need to supply code...Create a ...
- 4568 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Box_clown ,To be precise, Delta Lake format is based on parquet files. For strings, Parquet only has one data type: StringTypeSo, basically varchar(n) data type under the hood is represented as string with check constraint on the length of the st...
- 3 kudos
- 1643 Views
- 1 replies
- 1 kudos
RLS
When applying a function to a table for RLS, do users require SELECT privileges on the table used for RLS. And, do users also require EXECUTE privileges on the function. Not clear on this form doco.
- 1643 Views
- 1 replies
- 1 kudos
- 1 kudos
Yes, you require select permission for the table.For functions, if it is a built-in function (such as is_account_group_member), then you do not require permission. However, if it is a custom function, you must have access to execute it.You can refer ...
- 1 kudos
- 7161 Views
- 3 replies
- 0 kudos
- 7161 Views
- 3 replies
- 0 kudos
- 1008 Views
- 1 replies
- 0 kudos
Get queries history run on UC enabled interactive cluster
Hi Team,I want to derived couple of kpis like most frequent queries, top queries, query type like select, insert or update on UC enabled interactive cluster. I know we can do this for SQL warehouse but what is the way we can do this interactive clust...
- 1008 Views
- 1 replies
- 0 kudos
- 0 kudos
@Retired_mod , this table will only the query history for sql warehouse cluster, i need for UC interactive/All purpose cluster.
- 0 kudos
- 2561 Views
- 2 replies
- 3 kudos
Resolved! Service principal seemingly cannot access its own workspace folder
We have implemented an asset bundle (DAB) that creates a wheel. During DAB deployment, the wheel is built and stored in the folder of the service principal running the deployment via GH workflow. The full path is/Workspace/Users/SERVICE-PRINCIPAL-ID/...
- 2561 Views
- 2 replies
- 3 kudos
- 3 kudos
Thank you for sharing the solution that worked for you, I am sure it will help other community members. ThanksRishabh
- 3 kudos
- 9118 Views
- 3 replies
- 0 kudos
How to run a notebook in a .py file in databricks
The situation is that my colleague was using pycharm and now needs to adapt to databricks. They are now doing their job by connecting VScode to databricks and run the .py file using databricks clusters.The problem is they want to call a notebook in d...
- 9118 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Littlesheep_ , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your fe...
- 0 kudos
- 1352 Views
- 1 replies
- 0 kudos
How to extend the retention duration on steaming table created by DLT
The steaming table from DLT is default retention duration is 7 days. we would like to extend to 60 days. since we cannot alter the table properties, how can I achieve this change?
- 1352 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @EdwardLui , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...
- 0 kudos
- 2513 Views
- 2 replies
- 0 kudos
ConcurrentAppendException Liquid Clustered Table Different Row Concurrent Writes
I have multiple databricks jobs performing a MERGE command simultaneously into the same liquid clustered table but for different rows of data and I am receiving the following error message: [DELTA_CONCURRENT_APPEND] ConcurrentAppendException: Files w...
- 2513 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @georgecalvert , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your f...
- 0 kudos
- 1574 Views
- 1 replies
- 0 kudos
Delta live table not refreshing - window function
We have a list of streaming tables populated by Autoloader from files on S3, which serve as sources for our live tables. After the Autoloader Delta pipeline completes, we trigger a second Delta Live Tables (DLT) pipeline to perform a deduplication op...
- 1574 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @ibrar_aslam , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your fee...
- 0 kudos
- 2554 Views
- 2 replies
- 0 kudos
Resolved! [GCP] Failed to migrate a project onto an organization
Hi,After migrating a project to an organization, we are unable to create a workspace without encountering errors. Previously working workspaces are also failing.I have granted admin/owner access to all users who need Databricks. The latest error invo...
- 2554 Views
- 2 replies
- 0 kudos
- 0 kudos
The solution is...I had finaly edited constraints/iam.allowedPolicyMemberDomains on project and on organization with C01p0oudw (Databricks customer_id on GCP) and our customer_id (gcloud organizations describe YOUR_ORGANIZATION_ID). Not only on organ...
- 0 kudos
- 2263 Views
- 1 replies
- 2 kudos
Azure SCIM Usage and Alternatives for Databricks
Hello Databricks Community,I'm exploring the use of Azure SCIM for our Databricks environment and have a few questions:How is Azure SCIM useful for Databricks? What are the specific benefits or advantages of using SCIM for user and group provisioning...
- 2263 Views
- 1 replies
- 2 kudos
- 2 kudos
HI @jv_v , Thank you for reaching out to our community! We're here to help you. To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback n...
- 2 kudos
- 3004 Views
- 1 replies
- 0 kudos
Using spark jars using databricks-connect>=13.0
With the newest version of databricks-connect, I cannot configure the extra jars I want to use. In the older version, I did that viaspark = SparkSession.builder.appName('DataFrame').\ config('spark.jars.packages','org.apache.spark:spark-avro_...
- 3004 Views
- 1 replies
- 0 kudos
- 2689 Views
- 2 replies
- 0 kudos
Get run details of a databricks job that provides similar data without using api '/api/2.0/jobs/runs
I have a notebook, which is attached to a task at the end of a job. This task will pull the status of all other tasks in the job and checks if they are success or failure. Depending on the result, this last task will send a slack notification (custom...
- 2689 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @mannepk85 ,You can take a look on jobs system table. Notice though, that it is in public preview now so use it with caution: https://learn.microsoft.com/en-us/azure/databricks/admin/system-tables/jobs
- 0 kudos
- 2050 Views
- 1 replies
- 0 kudos
Get the external public IP of the Job Compute cluster
We just moved our workflow from "all purpose compute cluster" to "job compute cluster". We need to find out the external public IP of the Job Compute cluster. On the all purpose compute cluster, we get the IP by attaching a notebook and run the comm...
- 2050 Views
- 1 replies
- 0 kudos
- 0 kudos
I found the following IPs from the Cluster JSON file:"driver": {"private_ip": "10.*.*.*","public_dns": "172.*.*.*","node_id": "80*****",Similar the executors configuration"executors": [{"private_ip": "10.*.*.*","public_dns": "172.*.*.*","node_id": "7...
- 0 kudos
- 1542 Views
- 0 replies
- 0 kudos
Workflow scheduler cancel unreliable
Workflow paramtersWarning: 4m 30s | Timeout: 6m 50sThe jobs took 20-50 minutes to cancel.This workflow must have high reliability for our requirements. Does anyone know why the scheduler failed this morning at ~5:20 AM PT?After several failures, we'r...
- 1542 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 566 | |
| 349 | |
| 287 |
