I am getting the below error while Installing UCX. But Installation is done in the workspace.I have admin privileges on the workspace. The below error suggests to check token or URL of workspace.They are provided correctly.Then why below error is sho...
Hi all,cloning a devops repo worked like a charm every time, but now I get a weird error, that I do not understandWhat container is meant here and why would I need one?Thanks and regards,Raphael
@Yeshwanth I am sorry, I thought I replied to you on friday. I cannot upload a .har file. I get the error message:The file type (.har) is not supported. Valid file types are: jpg, gif, png, pdf.
Hi I have a unity catalog enabled workspaces in us-east-1 and would like to setup DR Databricks workspace in us-east-2. How can we achieve as I don't see a clear path for syncing metadata and delta tables across regions.
HiWe have a shared access mode cluster in which we have installed a maven library for reading excel files into a Spark DataFrame. When using an account with admin rights everything works fine, however when we run it as a standar user we always get `o...
Hello James, this issue seems to be related to shared cluster limitations, as per docs: Libraries used as JDBC drivers or custom Spark data sources on Unity Catalog-enabled shared compute require ANY FILE permissions.Moving to a single user cluster m...
Hi Team,we usually do setup profile for databrikcs at workspace level, and i have done this using host url and token which is working fine like below.[DEFAULT]host = workspaceurlusername = ****password = tokenNow my question is how we set for databri...
You cannot create a PAT token for you Account Console authentication, on this case you will have 2 options to authenticate:Using Basic auth, on which on your profile you will set you username and password to log in, same as you do to log in in the br...
I would like to add comments to all of our columns, programmatically. The only way I can find to do this is through SQL DDL - e.g. `alter table sometable alter column somecolumn comment 'some comment string`. However, I want to read this comment stri...
Hi @JameDavi_51481 ,I do not believe this is supported currently, please see this line in the Parameter Marker documentation: > You must not reference a parameter marker in a DDL statement...Ref: https://docs.databricks.com/en/sql/language-manual/sql...
I am working on use case to clone a table which has column and row level masking implemented.Can i clone the table, based on documentation it is not possible. what are the other alternatives i have ?
@amit-agarwal4 At the current moment, I'm not aware of any other way than disabling the RLS.
Hey folks,Facing an issue with zone_id not getting overridden when redeploying the DAB template to Databricks workspace.The Databricks job is already deployed and has "ap-south-1a" zone_id. I wanted to make it "auto" so, I have made the changes to th...
Hello,Thanks for the info I will try to figure it out for more.
Hello,after switching to "shared cluster" usage a python job is failing with error message: Py4JJavaError: An error occurred while calling o877.load. : org.apache.spark.SparkSecurityException: [INSUFFICIENT_PERMISSIONS] Insufficient privileges: User...
Hello @Hkesharwani ,thanks for replying.Indeed, as I stated in the beginning of my post, the issue occurs only with shared cluster usage (single user cluster all is fine). Since I *have to* switch to shared cluster (rowlevel security is only availabl...
Hi all,I am facing an issue with reactivating an inactive user i tried the following json with databricks cli run_update = { "schemas": [ "urn:ietf:params:scim:api:messages:2.0:PatchOp" ], "Operations": [ { "op": "replace", "path": "ac...
We are getting this same issue. Somehow two days ago two of our users got set to "Inactive". We've tried the scripts mentioned in the API but they will not reactivate.We also even got our Azure Owner to run the scripts and generate a pat to try, an...
Hi Team, I am trying to update the catalog permission privileges using databricks cli command Grant by appending json file but which is not updating the prIviliges, please help on grant update command usage.Command using : databricks grants update c...
Hello @Prasad_Koneru If the command is not updating the privileges as expected, there could be a few reasons for this. Firstly, ensure that the JSON file is correctly formatted and contains the correct privilege assignments. The privileges.json fi...
Databricks notebook, Scala, .read() .write(), source data csv. Got this error when trying to write data to sqlmi. I understand the error indicated one of the source column value's length exceeded the database column's value length. But the message is...
Thanks JamesY, I'm not familiar with the limitations of the SQL Server `nvarchar` data type but is there a way that we can filter out the rows that will fail using spark such as: spark.read.format("csv").option("header", "true").load("/path/to/csvs")...
When I'm running `dbt run -s model` for different models in parallel on databricks general compute cluster. Im getting the error: TABLE_OR_VIEW_ALREADY_EXISTS. Also, at the same time, the view/table was not created at all.But I run them in sequence, ...
@raphaelblg yes! Im the owner. However, these views are inside the hive_metastore catalog. Could it be the reason?Many thaaanks!
I am trying to install the stanza library and try to create a udf function to create NER tags for my chunk_text in the dataframe.Cluster Config: DBR 14.3 LTS SPARK 3.5.0 SCALA 2.12below code:def extract_entities(text import stanza nlp = stanza....
I have some CSV files that I upload to DBFS storage several times a day. From these CSVs, I have created SQL tables: CREATE TABLE IF NOT EXISTS masterdata.lookup_host USING CSV OPTIONS (header "true", inferSchema "true") LOCATION '/mnt/masterdata/...
One approach I'm testing (positive results so far, but still early).%sql # Prep and cleanup REFRESH TABLE masterdata.lookup_host; DROP TABLE IF EXISTS t_hosts; # Forcibly cache the needed columns before using the data in another query. CACHE TABLE...
