cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

fuselessmatt
by Contributor
  • 4514 Views
  • 4 replies
  • 1 kudos

Accidentally removing the service principal that owns the view seems to put the Unity Catalog in an illegal state. Can you fix this?

I renamed our service principal in Terraform, which forces a replacement where the old service principal is removed and a new principal with the same permission is recreated. The Terraform succeeds to apply, but when I try to run dbt that creates tab...

  • 4514 Views
  • 4 replies
  • 1 kudos
Latest Reply
fuselessmatt
Contributor
  • 1 kudos

This is also true for removing groups before unassigning them (removing and unassigning in Terraform)│ Error: cannot update grants: Could not find principal with name <My Group Name>

  • 1 kudos
3 More Replies
n-riesco
by New Contributor
  • 17896 Views
  • 5 replies
  • 0 kudos

How can I view an exported DBC notebook in my computer?

Is it possible to convert to or export as a .ipynb notebook?

  • 17896 Views
  • 5 replies
  • 0 kudos
Latest Reply
AlexV
New Contributor II
  • 0 kudos

You can rename somefile.dbc to somefile.zip and open it with the Windows File Explorer, however the .python files cannot be opened in vscode or pycharm

  • 0 kudos
4 More Replies
RobiTakToRobi
by New Contributor II
  • 1152 Views
  • 1 replies
  • 1 kudos

How to allow non-ASCII characters to be stored in the view definition?

I've tried to create a view with a simple conditional statement containing Polish characters. The view is created without errors, but select on the view returns question marks in place of the non-ASCII characters. Why? How to fix it?Below on screens ...

example view_text
  • 1152 Views
  • 1 replies
  • 1 kudos
Latest Reply
andreas7891
New Contributor II
  • 1 kudos

Any solutions on this?We have the same problem with Greek characters.

  • 1 kudos
cmilligan
by Contributor II
  • 4142 Views
  • 1 replies
  • 0 kudos

Pull query that inserts into table

I'm trying to pull some data down for table history and am needing to view the query that inserted into a table. My team owns the process so I'm able to view the current query by just viewing it but I'm also wanting to capture changes over time witho...

  • 4142 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Coleman Milligan​ :Yes, in Databricks, you can use the built-in Delta Lake feature to track the history of changes made to a table, including the queries that inserted data into it.Here's an example of how to retrieve the queries that inserted data ...

  • 0 kudos
zyang
by Contributor
  • 6089 Views
  • 2 replies
  • 1 kudos

Set owner when creating a view in databricks sql

Hi,I would like to set the owner when create a view in databricks sql.Is it possible? I cannot find anything about it.Best

  • 6089 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @z yang​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your q...

  • 1 kudos
1 More Replies
najmead
by Contributor
  • 2555 Views
  • 2 replies
  • 0 kudos

Creating an external table reference vs creating a view

In a practical sense, what is the difference between creating an external table;create table my_catalog.my_schema.my_favourite_table location 'abfss://path/to/my/dataversus creating a view that references the same dataset;create view my_catalog.my_sc...

  • 2555 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Nicholas Mead​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...

  • 0 kudos
1 More Replies
oleole
by Contributor
  • 8236 Views
  • 1 replies
  • 1 kudos

Resolved! MERGE to update a column of a table using Spark SQL

Coming from MS SQL background, I'm trying to write a query in Spark SQL that simply update a column value of table A (source table) by INNER JOINing a new table B with a filter.MS SQL query looks like this:UPDATE T SET T.OfferAmount = OSE.EndpointEve...

  • 8236 Views
  • 1 replies
  • 1 kudos
Latest Reply
oleole
Contributor
  • 1 kudos

Posting answer to my question:   MERGE into TempOffer VIEW USING OfferSeq OSE ON VIEW.OfferId = OSE.OfferID AND OSE.OfferId = 1 WHEN MATCHED THEN UPDATE set VIEW.OfferAmount = OSE.EndpointEventAmountValue;

  • 1 kudos
andrew0117
by Contributor
  • 2012 Views
  • 2 replies
  • 0 kudos

depth of view exceeds the maximum view resolution depth (100).

I got this error after updating a view. How can I increase the value of spark.sql.view.maNestedViewDepth to work around this? Thanks!

  • 2012 Views
  • 2 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Could you please confirm if you are showing the view? (https://docs.databricks.com/sql/language-manual/sql-ref-syntax-aux-show-views.html) also, it will be helpful if you post the screenshot of the error.

  • 0 kudos
1 More Replies
Nhan_Nguyen
by Valued Contributor
  • 1313 Views
  • 1 replies
  • 6 kudos

Resolved! Logic execute when we create an View?

Hi all,I have a small curious about VIEW on Databricks. Could anyone please help me clarify this?Normal database like Postgres or MS SQL, when we define a view, the logic still not execute that time, only run when we query that VIEW.Not sure how VIEW...

  • 1313 Views
  • 1 replies
  • 6 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 6 kudos

@Nhan Nguyen​ It works the same. CREATE VIEW constructs a virtual table that has no physical data.

  • 6 kudos
fury88
by New Contributor II
  • 1285 Views
  • 1 replies
  • 1 kudos

Does CACHE TABLE/VIEW have a create or replace like view?

I'm trying to cache data/queries that we normally have as temporary views that get replaced when the code is run based on dynamic python. What I'd like to know is will CACHE TABLE get overwritten each time you run it? Is it smart enough to recognize ...

  • 1285 Views
  • 1 replies
  • 1 kudos
Latest Reply
UmaMahesh1
Honored Contributor III
  • 1 kudos

Hi @Matt Fury​ Yes...I guess cache overwrites each time you run it because for me it took nearly same amount of time for 1million records to be cached. However, you can check whether the table is cached or not using .storageLevel method. E.g. I have...

  • 1 kudos
mickniz
by Contributor
  • 3262 Views
  • 3 replies
  • 6 kudos

Unable to create materialized view in Unity enabled Catalogues.

Hi Team,I was going through one of the videos of Databricks Sql Serverless and it say there is materialized view support . We can create materialized view .I tried same on my cluster of Sql Warehouse gives below error: 

image
  • 3262 Views
  • 3 replies
  • 6 kudos
Latest Reply
Felix
New Contributor III
  • 6 kudos

Materialized views is in private preview right now afaik. Please talk to your account or customer success team at Databricks in order to sign up and enable it for your workspace. Thanks!

  • 6 kudos
2 More Replies
sage5616
by Valued Contributor
  • 5986 Views
  • 12 replies
  • 10 kudos

Error in SQL statement: AnalysisException: Cannot up cast documents from array

Hi Everyone,I am getting the following error when running a SQL query and do not understand what it means or what can be done to resolve it. Any recommendations?View DDL:CREATE VIEW myschema.table ( accountId, agreementType, capture_file_name, ...

  • 5986 Views
  • 12 replies
  • 10 kudos
Latest Reply
Anonymous
Not applicable
  • 10 kudos

Hi @Michael Okulik​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 10 kudos
11 More Replies
elgeo
by Valued Contributor II
  • 1240 Views
  • 1 replies
  • 1 kudos

Merge didn't fail while inserting wrong data type values

Hello. During some example cases we were running, in order to identify how Databricks treats possible wrong actions we could make, we noticed that merge doesn't fail while inserting different data type values from the ones in the corresponding table....

Target table schema merge_failure image merge_failure3
  • 1240 Views
  • 1 replies
  • 1 kudos
Latest Reply
elgeo
Valued Contributor II
  • 1 kudos

Hello. Any update on this please? Thank you in advance

  • 1 kudos
ekdz__
by New Contributor III
  • 3952 Views
  • 5 replies
  • 10 kudos

Is there any way to save the notebook in the "Results Only" view?

Hi! I'm looking for a solution to save a notebook in HTML format that has the "Results Only" view (without the executed code). Is there any possibility to do that?Thank you

  • 3952 Views
  • 5 replies
  • 10 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 10 kudos

Hi @Eryk Kądziela​, We haven’t heard from you on the last response from @hubert, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please share it with the community as it can be helpful to others.

  • 10 kudos
4 More Replies
User16790091296
by Contributor II
  • 12261 Views
  • 6 replies
  • 1 kudos

How to delete from a temp view or equivalent in spark sql databricks?

I need to delete from a temp view in databricks, but it looks like i can do only merge, select and insert. Maybe i missed something but I did not find any documentation on this.

  • 12261 Views
  • 6 replies
  • 1 kudos
Latest Reply
crazy_horse
New Contributor II
  • 1 kudos

What about%sqlDROP TABLE IF EXISTS xxxxx

  • 1 kudos
5 More Replies
Labels