- 3785 Views
- 2 replies
- 0 kudos
Accessing ADLS Gen2 related Hadoop configuration in notebook
I have a cluster in which I have the required configuration to access an ADLS Gen2, and it works without any problems.I want to access this storage using the Hadoop filesystem APIs. To achieve this, I am trying to get the Hadoop configuration from th...
- 3785 Views
- 2 replies
- 0 kudos
- 0 kudos
By the way how do you achieve inline code highlighting in the editor I tried `` but it didn't work.
- 0 kudos
- 2142 Views
- 1 replies
- 2 kudos
Resolved! DAIS24 Attendee Badge - Oops
I wasn't at DAIS24, but I received one of these emails and it appears to have come from Databricks. I think I was given the badge in error. Can we have it removed?Just Wondering,Shawn
- 2142 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Shawn_Eary Thank you for bringing this to our attention. We were conducting a test on badges, and some of our community members may have received this email by accident. We have revoked the changes now, please ignore the message.
- 2 kudos
- 3346 Views
- 1 replies
- 0 kudos
Tableau Prep Save Output to Databricks
Has anyone run into use cases where your data scientist/data engineer end users build Tableau Prep Flows and steps in Tableau Prep Flow require saving output back into Databricks? There appears to be no native support for this in Tableau Prep if the ...
- 3346 Views
- 1 replies
- 0 kudos
- 0 kudos
These are awesome suggestions. To expand on our setup, we also have Informatica Cloud - IICS (CMI, CDI, etc.) connected to the entire setup generally used for bringing data from a source (PaaS, SaaS, On-prem SQL, Flat Files or streaming devices) to D...
- 0 kudos
- 4786 Views
- 4 replies
- 1 kudos
Resolved! Getting error while running the MLOPS demo
Hello,I am trying to run this demo in databricks community edition but i am facing error.MLOPS DEMO - https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/mlops-end-to-end-pipeline?itm_data=demo_centerSomeone else also faced the s...
- 4786 Views
- 4 replies
- 1 kudos
- 1 kudos
Hey @Kushal_Nagrani , thank you for sharing this. I am facing the same issue. Could you please help me where can I find these settings? I am new, just created a community workspace and I thought given I am the admin I will have all these by default -...
- 1 kudos
- 1306 Views
- 0 replies
- 0 kudos
Training link deprecated: How to ingest data for Databricks SQL
I am currently doing a course in Databricks academy: How to ingest data for Databricks SQL.To create a table in the external location I am provided with the link that is not working anymore. Below is the link:wasbs://courseware@dbacademy.blob.core.wi...
- 1306 Views
- 0 replies
- 0 kudos
- 1971 Views
- 1 replies
- 0 kudos
Materialized Views Still In Public Preview?
The title is self-explanatory, but I'm curious if anyone has any inclination as to when Materialized Views will enter General Availability? I'm surprised it's not already in GA considering (A) it's been a year or more since Databricks originally anno...
- 1971 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, if the documentation states it, it's still in preview.
- 0 kudos
- 8596 Views
- 2 replies
- 0 kudos
Seeking Tips: Ways to Master Databricks on Azure?
Hello everyone. I'm currently learning Databricks on Azure through a Udemy course. Recently, I was surprised by a charge of $86 from Azure, which has made me cautious about continuing in this manner. Is there a more cost-effective approach to learn D...
- 8596 Views
- 2 replies
- 0 kudos
- 3596 Views
- 1 replies
- 0 kudos
databricks asset bundles on azure devops
@herewe are able to work on DAB locally or Github actions, but looks from azure devops end lot of missing pieces in terms of pipeline and examples. can any one point me out github repo that is good to test azure devops pipeline for DAB.
- 3596 Views
- 1 replies
- 0 kudos
- 1512 Views
- 1 replies
- 0 kudos
Assigning a group as USER to service principal
How can we assigning a group as USER to service principal using databricks-sdk, this is not supported?
- 1512 Views
- 1 replies
- 0 kudos
- 0 kudos
I found this API, https://docs.databricks.com/api/account/accountaccesscontrol/updaterulesetbut its PUT and GET method both requires a parameter "etag", how can someone know this "etag"?
- 0 kudos
- 3161 Views
- 3 replies
- 2 kudos
DLT Compute: "Ephemeral" Job Compute vs. All-purpose compute
Hi there, I would like to understand for DLT jobs if there is any way to get DLT jobs running in an *existing* (and currently running) All-purpose compute rather than spinning up an "ephemeral" (not yet initialized) Job Compute?
- 3161 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @ChristianRRL , you cannot run a DLT pipeline in an All-purpose compute cluster. DLT Clusters are managed resources. Here are some details on how to configure a DLT cluster: https://docs.databricks.com/en/delta-live-tables/settings.html#configure-...
- 2 kudos
- 6967 Views
- 3 replies
- 0 kudos
List granted access for a group or a user
Is there any way where I can see what access a group or a user have been given to objects (Tables, views, catalogs etc.)?I noticed that we have the following information_schema tables:catalog_privilegesroutine_privilegestable_privilegesschema_privile...
- 6967 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Henrik To grant a user the privilege to query system tables, a metastore admin or another privileged user must grant USE and SELECT permissions on the system schemas. GRANT USAGE ON CATALOG system TO <user_name>; GRANT USAGE ON SCHEMA informa...
- 0 kudos
- 20464 Views
- 3 replies
- 3 kudos
org.apache.spark.SparkException: Job aborted due to stage failure:
HiI have around 20 million records in my DF, and want to save it in HORIZINTAL SQL DB.This is error:org.apache.spark.SparkException: Job aborted due to stage failure: A shuffle map stage with indeterminate output was failed and retried. However, Spar...
- 20464 Views
- 3 replies
- 3 kudos
- 3 kudos
If there are any failures which may lead to a stage retry, but retrying the stage translates into potentially having an inconsistent result (indeterminacy) then this exception is raised. The exception is raised in newer version where the validation i...
- 3 kudos
- 2562 Views
- 0 replies
- 0 kudos
Databricks + confluent schema registry - Schema not found error
I am running a Kafka producer code on Databricks 12.2. I am testing AVRO serialization of message with help of confluent schema registry. I configured 'to_avro' function to read the schema from schema registry, but I am getting the below error> org.a...
- 2562 Views
- 0 replies
- 0 kudos
- 1548 Views
- 0 replies
- 0 kudos
Identifying the Dependency Automatically on the SQL Statements
Hi All,We have a SQL script file with lots of create, insert, and SQL statements. We need to figure out how to identify the dependencies between the statements dynamically(by using either spark/sql or using any program).For example - First, we need t...
- 1548 Views
- 0 replies
- 0 kudos
- 2019 Views
- 1 replies
- 0 kudos
Materialized view in DLT pipeline
When setting up DLT pipeline, there are 3 types of product edition, Core, Pro and Advanced. When I compare DLT Classic Core and DLT Classic Pro, the difference is that DLT Classic Pro can handle CDC. Does it means if I'm using DLT Classic Core, I hav...
- 2019 Views
- 1 replies
- 0 kudos
- 0 kudos
Also, If I'm using DLT Classic Core for DLT pipeline, the materialized view will be doing a full refresh or just update on the rows that have changes?
- 0 kudos
