- 13832 Views
- 12 replies
- 3 kudos
Resolved! Unity catalog - How do you modify groups properly ?
Hello,What is the best practice to modify/delete/recreate groups properly ?In order to rename a group, the only mean was to delete/recreate. But after deletion in the account console, the permissions granted to the deleted groups in the tables were i...
- 13832 Views
- 12 replies
- 3 kudos
- 3 kudos
Hello,I have exactly the same issue - I am also using terraform.I deleted a group and the catalog permissions are in bad state. I am not able to revoke access to this group using the Databricks UI nor REST API. I also tried to recreate the group wit...
- 3 kudos
- 3200 Views
- 1 replies
- 2 kudos
Liquid Clustering - Number of files are increasing
We enabled liquid clustering on one of the large tables (380GBs). This table goes many operations daily, which improved many folds after liquid clustering. However, after enabling liquid clustering and optimizing it number of files are increased.Prev...
- 3200 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for detailed explanation @Retired_mod .
- 2 kudos
- 4779 Views
- 1 replies
- 1 kudos
Resolved! Asset Bundles : how to conditionally set content of a template file
Hello,since Asset Bundles is based on GO templating mechanism, I am wondering how it is possible to use IF-ELSE construct within a template file, to define which file content will be set in the generated file ( I want to have that in my custom templa...
- 4779 Views
- 1 replies
- 1 kudos
- 1 kudos
Never mind....I figured out the solution I just have to prefix my template file with ".tmpl", then it gets rendered correctly
- 1 kudos
- 2519 Views
- 4 replies
- 0 kudos
Databricks Job Params
Hi,Job params override the task params(same name params). Is there a way task params override the job params.Use case:job params: a = "param-1".job has 12 tasks. 10 of them should use job param(a = "param-1").2 of them should override the job param(a...
- 2519 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @ajithgaade,Unfortunately, I don't think it is currently possible. It's clearly stated in documentation that:" Job parameters take precedence over task parameters. If a job parameter and a task parameter have the same key, the job parameter overri...
- 0 kudos
- 2025 Views
- 1 replies
- 0 kudos
Failure to Create Materialize View - Error Starting DLT Service on Databricks Cluster xxxx-xxxxxx-xx
Hi,I am trying to create a materialized view; however, it failed. I could not find any reason for this issue.Cluster Summary:Cluster Type: SQL warehouse serverless with Unity CatalogWorkers: 1-4 Workers (16-64 GB Memory, 4-16 Cores)Driver: 1 Driver ...
- 2025 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @taimoor123 , could you share the stack trace for the error please ? Also, I find it a bit confusing. You mention you are using a Serverless SQL warehouse to run the query, but you are also providing the details of the configuration of what seem...
- 0 kudos
- 2536 Views
- 3 replies
- 2 kudos
Data Lineage with Apply Changes
Hello TeamI am using DLT. I am able to see the lineage when doing normal process. However as soon as I use 'APPLY_Changes' feature after the lineag ebreaks and I am no more able to see the Data lineage from the catalog after going to table. Is there ...
- 2536 Views
- 3 replies
- 2 kudos
- 2 kudos
@Retired_mod : Is there any way this can be achieved or can we expect this problem to be resolved in next releases?
- 2 kudos
- 3598 Views
- 2 replies
- 0 kudos
Resolved! Init Script Fails Intermittently on Workflow Job
An init script is used to install system libraries, per below.Adding the script to a Personal Compute consistently works. The same script is added to a Workflows job via cluster config, which intermittently fails, as shown in error message below.Both...
- 3598 Views
- 2 replies
- 0 kudos
- 0 kudos
Check the cluster event log to see if there is a clue why the script is failing. if the script failed and returned none zero status the cluster wont start
- 0 kudos
- 2716 Views
- 1 replies
- 0 kudos
Resolved! Overwriting mode do not overwrite
I have the following codePreviously I have a delta table with 180 columns in my_path´, I select a column and try to overwrite columns_to_select = ["one_column"] df_one_column = df.select(*columns_to_select) df_one_column.write.format("de...
- 2716 Views
- 1 replies
- 0 kudos
- 0 kudos
ok I get the Issue .option("mergeSchema", "true") Is usefull to add more columns, but if you want to reduce columns in your target delta.Then you need.option("overwriteSchema", "true")
- 0 kudos
- 876 Views
- 1 replies
- 0 kudos
Metastore Access
Hi team,We are operating on a single-tenant basis with just one metastore. How can other teams use Databricks Unity Catalog without being granted access to our metadata within single tenantRegards,Janga
- 876 Views
- 1 replies
- 0 kudos
- 0 kudos
You can enable/set workspace catalog isolation for each catalog used by your teams. That way, they'll only be able to see their own catalog and corresponding data assets.
- 0 kudos
- 2425 Views
- 4 replies
- 0 kudos
Databricks API - Create Connection
Hi,Is it possible to use the Databricks API to create a connection to a different Azure Databricks workspace?Thanks
- 2425 Views
- 4 replies
- 0 kudos
- 0 kudos
My apologies for not being cleared in the first post.Terraform can handle it. The API is probably behind and can't handle the connection to Databricks yet. The best solution would probably be to just move to Unity Catalog...Thanks for your quick repl...
- 0 kudos
- 4121 Views
- 5 replies
- 7 kudos
Resolved! Change Runtime version from 10.4 to 11.3 in ADF
Hello! At the moment, I am considering replacing runtime 10.4 with runtime 11.3 in my ADF. I would like to know how big the differences are between them, and if I can just change the version without worrying about breaking anything?
- 4121 Views
- 5 replies
- 7 kudos
- 7 kudos
On thing worth mentioning is the DBR Migration Guide. There you'll find all potential changes between the runtimes.
- 7 kudos
- 1427 Views
- 1 replies
- 0 kudos
OutputMode “complete” unable to replace the entire table
According to the document https://docs.databricks.com/en/structured-streaming/delta-lake.html#complete-mode, the “complete” option seems to “replace the entire table with every batch”. However, it is not working in my case.Here is how I reproduce the...
- 1427 Views
- 1 replies
- 0 kudos
- 0 kudos
I figure out it already. I cannot find the delete button. Please ignore this post
- 0 kudos
- 1943 Views
- 1 replies
- 1 kudos
Resolved! Deep Clone
Will a deep clone of a table also clone the table's tags?
- 1943 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @acegerace , By default, table Tags are not included in the cloned metadata. This can be observed by cloning the table and confirming in the `information_schema.table_tags` system table. If this is not the desired behavior, the tags could be set o...
- 1 kudos
- 4730 Views
- 1 replies
- 1 kudos
structured streaming hangs when writing or sometimes reading depends on SINGLE USER or shared mode
Hi Guys,I'm new to this community, I am beginning a new project with Azure Databricks and a Python script on my Mac that manipulates data (reading from delta share tables and inserting into local Postgres database ) coming from a remote databricks cl...
- 4730 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi,I reckon I've seen this problem earlier and if I remember correctly there was an issue related to Databricks Connect and single user mode.I think your code will work if you use it directly in notebook, but will fail with Databricks Connect.About t...
- 1 kudos
- 4223 Views
- 3 replies
- 1 kudos
SQL Server Connection on LTS 14.3
Hi, what is the best way to connect to a SQL Server on LTS 14.3?I've been trying to setup a connection to an SQL Server as referenced here. However both using both "sqlserver" and "jdbc" as format has resulted in an exception when using display as sp...
- 4223 Views
- 3 replies
- 1 kudos
- 1 kudos
Bit weird, I've tested option with format('sqlserver') and I didn't notice any problems on LTS 14.3. Did you check what happens if you use databricks specific display command, like so:display(df)
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 584 | |
| 349 | |
| 287 |
