cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

claudiazi
by New Contributor II
  • 1895 Views
  • 5 replies
  • 1 kudos

Resolved! [TABLE_OR_VIEW_ALREADY_EXISTS] when running create or replace view

When I'm running `dbt run -s model` for different models in parallel on databricks general compute cluster. Im getting the error: TABLE_OR_VIEW_ALREADY_EXISTS. Also, at the same time, the view/table was not created at all.But I run them in sequence, ...

  • 1895 Views
  • 5 replies
  • 1 kudos
Latest Reply
claudiazi
New Contributor II
  • 1 kudos

@raphaelblg yes! Im the owner. However, these views are inside the hive_metastore catalog. Could it be the reason?Many thaaanks!

  • 1 kudos
4 More Replies
hukel
by Contributor
  • 2682 Views
  • 2 replies
  • 0 kudos

InconsistentReadException: The file might have been updated during query - CSV backed table

I have some CSV files that I upload to DBFS storage several times a day.   From these CSVs,  I have created SQL tables: CREATE TABLE IF NOT EXISTS masterdata.lookup_host USING CSV OPTIONS (header "true", inferSchema "true") LOCATION '/mnt/masterdata/...

  • 2682 Views
  • 2 replies
  • 0 kudos
Latest Reply
hukel
Contributor
  • 0 kudos

One approach I'm testing (positive results so far, but still early).%sql # Prep and cleanup REFRESH TABLE masterdata.lookup_host; DROP TABLE IF EXISTS t_hosts; # Forcibly cache the needed columns before using the data in another query. CACHE TABLE...

  • 0 kudos
1 More Replies
rodrigosanchesz
by New Contributor II
  • 6013 Views
  • 0 replies
  • 0 kudos

Change adls gen2 attached to Unity Catalog metastore from premium to standard

Hello,Our cloud platform engineer created Azure storage for the production Unity Catalog metastore in our environment, but mistakenly chose the Premium tier instead of the Standard tier.Unfortunately, this decision is impacting our costs on Azure, as...

  • 6013 Views
  • 0 replies
  • 0 kudos
Ender
by New Contributor II
  • 1502 Views
  • 2 replies
  • 0 kudos

Accessing ADLS Gen2 related Hadoop configuration in notebook

I have a cluster in which I have the required configuration to access an ADLS Gen2, and it works without any problems.I want to access this storage using the Hadoop filesystem APIs. To achieve this, I am trying to get the Hadoop configuration from th...

Ender_1-1717335940727.png Ender_3-1717336689720.png
  • 1502 Views
  • 2 replies
  • 0 kudos
Latest Reply
Ender
New Contributor II
  • 0 kudos

By the way how do you achieve inline code highlighting in the editor I tried `` but it didn't work.

  • 0 kudos
1 More Replies
Shawn_Eary
by Contributor
  • 694 Views
  • 1 replies
  • 2 kudos

Resolved! DAIS24 Attendee Badge - Oops

I wasn't at DAIS24, but I received one of these emails and it appears to have come from Databricks. I think I was given the badge in error. Can we have it removed?Just Wondering,Shawn

Shawn_Eary_1-1717680651702.png
  • 694 Views
  • 1 replies
  • 2 kudos
Latest Reply
Sujitha
Databricks Employee
  • 2 kudos

Hi @Shawn_Eary Thank you for bringing this to our attention. We were conducting a test on badges, and some of our community members may have received this email by accident. We have revoked the changes now, please ignore the message.

  • 2 kudos
egndz
by New Contributor II
  • 4814 Views
  • 1 replies
  • 0 kudos

Cluster Memory Issue (Termination)

Hi,I have a single-node personal cluster with 56GB memory(Node type: Standard_DS5_v2, runtime: 14.3 LTS ML). The same configuration is done for the job cluster as well and the following problem applies to both clusters:To start with: once I start my ...

egndz_2-1712845742934.png egndz_1-1712845616736.png
  • 4814 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hi,I have a single-node personal cluster with 56GB memory(Node type: Standard_DS5_v2, runtime: 14.3 LTS ML). The same configuration is done for the job cluster as well and the following problem applies to both clusters:To start with: once I start my ...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
vabadzhiev
by New Contributor II
  • 1132 Views
  • 1 replies
  • 0 kudos

Tableau Prep Save Output to Databricks

Has anyone run into use cases where your data scientist/data engineer end users build Tableau Prep Flows and steps in Tableau Prep Flow require saving output back into Databricks? There appears to be no native support for this in Tableau Prep if the ...

  • 1132 Views
  • 1 replies
  • 0 kudos
Latest Reply
vabadzhiev
New Contributor II
  • 0 kudos

These are awesome suggestions. To expand on our setup, we also have Informatica Cloud - IICS (CMI, CDI, etc.) connected to the entire setup generally used for bringing data from a source (PaaS, SaaS, On-prem SQL, Flat Files or streaming devices) to D...

  • 0 kudos
kazinahian
by New Contributor III
  • 7215 Views
  • 2 replies
  • 0 kudos

Seeking Tips: Ways to Master Databricks on Azure?

Hello everyone. I'm currently learning Databricks on Azure through a Udemy course. Recently, I was surprised by a charge of $86 from Azure, which has made me cautious about continuing in this manner. Is there a more cost-effective approach to learn D...

  • 7215 Views
  • 2 replies
  • 0 kudos
Latest Reply
LauraMurphy
New Contributor II
  • 0 kudos

Thank you so much for the information.

  • 0 kudos
1 More Replies
samarth10
by New Contributor II
  • 464 Views
  • 1 replies
  • 0 kudos

Assigning a group as USER to service principal

How can we assigning a group as USER to service principal using databricks-sdk, this is not supported?

  • 464 Views
  • 1 replies
  • 0 kudos
Latest Reply
samarth10
New Contributor II
  • 0 kudos

I found this API, https://docs.databricks.com/api/account/accountaccesscontrol/updaterulesetbut its PUT and GET method both requires a parameter "etag", how can someone know this "etag"?

  • 0 kudos
Avvar2022
by Contributor
  • 3353 Views
  • 6 replies
  • 2 kudos

Unity catalog enabled workspace -Is there any way to disable workflow/job creation for certain users

Currently in unity catalog enabled workspace users with "Workspace access" can create workflows/jobs, there is no access control available to restrict users from creating jobs/workflows.Use case: In production there is no need for users, data enginee...

  • 3353 Views
  • 6 replies
  • 2 kudos
Latest Reply
Avvar2022
Contributor
  • 2 kudos

not being able to restrict creation of workflows/jobs, alerts, dashboards make platform admin job difficult to keep system clean and control cost. there is no need for data engineers to create a workflow in production. there is no need for all users ...

  • 2 kudos
5 More Replies
Henrik
by New Contributor III
  • 1090 Views
  • 3 replies
  • 0 kudos

List granted access for a group or a user

Is there any way where I can see what access a group or a user have been given to objects (Tables, views, catalogs etc.)?I noticed that we have the following information_schema tables:catalog_privilegesroutine_privilegestable_privilegesschema_privile...

  • 1090 Views
  • 3 replies
  • 0 kudos
Latest Reply
anardinelli
Databricks Employee
  • 0 kudos

Hi @Henrik  To grant a user the privilege to query system tables, a metastore admin or another privileged user must grant USE and SELECT permissions on the system schemas.   GRANT USAGE ON CATALOG system TO <user_name>; GRANT USAGE ON SCHEMA informa...

  • 0 kudos
2 More Replies
Kasen
by New Contributor III
  • 789 Views
  • 1 replies
  • 0 kudos

Materialized view in DLT pipeline

When setting up DLT pipeline, there are 3 types of product edition, Core, Pro and Advanced. When I compare DLT Classic Core and DLT Classic Pro, the difference is that DLT Classic Pro can handle CDC. Does it means if I'm using DLT Classic Core, I hav...

Community Platform Discussions
DLT pipeline
Materialized view
  • 789 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kasen
New Contributor III
  • 0 kudos

Also, If I'm using DLT Classic Core for DLT pipeline, the materialized view will be doing a full refresh or just update on the rows that have changes? 

  • 0 kudos
SamGreene
by Contributor
  • 1082 Views
  • 0 replies
  • 0 kudos

String to date conversion errors

Hi,I am getting data from CDC on SQL Server using Informatica which is writing parquet files to ADLS.  I read the parquet files using DLT and end up with the date data as a string such as this'20240603164746563' I couldn't get this to convert using m...

  • 1082 Views
  • 0 replies
  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors