by
alm
• New Contributor III
- 7779 Views
- 6 replies
- 2 kudos
I have a medallion architecture: Bronze layer: Raw data in tablesSilver layer: Refined data in views created from the bronze layerGold layer: Data products as views created from the silver layerCurrently I have a data scientist that needs access to d...
- 7779 Views
- 6 replies
- 2 kudos
Latest Reply
Single-user clusters use a different security mode which is the reason for this difference.
On single-user/assigned clusters, you'll need the Fine Grained Access Control service (which is a Serverless service) - that is the solution to this problem (...
5 More Replies
- 6107 Views
- 2 replies
- 1 kudos
In GCP you can give a user access to a view, and then the view itself access to the underlying object, meaning you don't have to give end users access to the tables themselves.Is there a similar way of managing these permissions in databricks? The vi...
- 6107 Views
- 2 replies
- 1 kudos
- 792 Views
- 0 replies
- 0 kudos
Currently we are facing an issue with global temp views. We create in one notebook a global temp view and query the results in another notebook on the same cluster. The issue is, that the result is not consistent. When we query the view several times...
- 792 Views
- 0 replies
- 0 kudos
by
bdugar
• New Contributor II
- 17848 Views
- 1 replies
- 2 kudos
Hi:It's possible to create temp views in pyspark using a dataframe (df.createOrReplaceTempView()), and it's possible to create a permanent view in Spark SQL. But as far as I can tell, there is no way to create a permanent view from a dataframe, somet...
- 17848 Views
- 1 replies
- 2 kudos
Latest Reply
Hi Kaniz:This is what I understood from the research I did, I was curious more as to why permanent views can't be created from dataframes and whether this is a feature that might be implemented by Databricks or Spark at some point. Temporary views ca...
- 20870 Views
- 1 replies
- 6 kudos
So I'm querying data from parquet files that have a couple of billions records (table 1 or t1), and then have to filter and then join with other parquet files with another couple of billions records (t2). This takes quite a long time to run (like 10h...
- 20870 Views
- 1 replies
- 6 kudos
Latest Reply
Your intuition about views is correct. Views are not materialized, so they are basically just a saved query. Every time you access a view it will have to be recomputed. This is certainly not ideal if it take a long time (like 10hrs) to materialize a ...
- 18304 Views
- 1 replies
- 1 kudos
Is there general guidance around using views vs creating Delta tables? For example, I need to do some filtering and make small tweaks to a few columns for use in another application. Is there a downside of using a view here?
- 18304 Views
- 1 replies
- 1 kudos
Latest Reply
Views won't duplicate the data so if you are just filtering columns or rows or making small tweaks then views might be a good option. Unless, of course, the filtering is really expensive or you are doing a lot of calculations, then materialize the vi...
