- 4451 Views
- 4 replies
- 1 kudos
Resolved! Getting error while running the MLOPS demo
Hello,I am trying to run this demo in databricks community edition but i am facing error.MLOPS DEMO - https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/mlops-end-to-end-pipeline?itm_data=demo_centerSomeone else also faced the s...
- 4451 Views
- 4 replies
- 1 kudos
- 1 kudos
Hey @Kushal_Nagrani , thank you for sharing this. I am facing the same issue. Could you please help me where can I find these settings? I am new, just created a community workspace and I thought given I am the admin I will have all these by default -...
- 1 kudos
- 1175 Views
- 0 replies
- 0 kudos
Training link deprecated: How to ingest data for Databricks SQL
I am currently doing a course in Databricks academy: How to ingest data for Databricks SQL.To create a table in the external location I am provided with the link that is not working anymore. Below is the link:wasbs://courseware@dbacademy.blob.core.wi...
- 1175 Views
- 0 replies
- 0 kudos
- 1805 Views
- 1 replies
- 0 kudos
Materialized Views Still In Public Preview?
The title is self-explanatory, but I'm curious if anyone has any inclination as to when Materialized Views will enter General Availability? I'm surprised it's not already in GA considering (A) it's been a year or more since Databricks originally anno...
- 1805 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, if the documentation states it, it's still in preview.
- 0 kudos
- 8294 Views
- 2 replies
- 0 kudos
Seeking Tips: Ways to Master Databricks on Azure?
Hello everyone. I'm currently learning Databricks on Azure through a Udemy course. Recently, I was surprised by a charge of $86 from Azure, which has made me cautious about continuing in this manner. Is there a more cost-effective approach to learn D...
- 8294 Views
- 2 replies
- 0 kudos
- 3456 Views
- 1 replies
- 0 kudos
databricks asset bundles on azure devops
@herewe are able to work on DAB locally or Github actions, but looks from azure devops end lot of missing pieces in terms of pipeline and examples. can any one point me out github repo that is good to test azure devops pipeline for DAB.
- 3456 Views
- 1 replies
- 0 kudos
- 1363 Views
- 1 replies
- 0 kudos
Assigning a group as USER to service principal
How can we assigning a group as USER to service principal using databricks-sdk, this is not supported?
- 1363 Views
- 1 replies
- 0 kudos
- 0 kudos
I found this API, https://docs.databricks.com/api/account/accountaccesscontrol/updaterulesetbut its PUT and GET method both requires a parameter "etag", how can someone know this "etag"?
- 0 kudos
- 2940 Views
- 3 replies
- 2 kudos
DLT Compute: "Ephemeral" Job Compute vs. All-purpose compute
Hi there, I would like to understand for DLT jobs if there is any way to get DLT jobs running in an *existing* (and currently running) All-purpose compute rather than spinning up an "ephemeral" (not yet initialized) Job Compute?
- 2940 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @ChristianRRL , you cannot run a DLT pipeline in an All-purpose compute cluster. DLT Clusters are managed resources. Here are some details on how to configure a DLT cluster: https://docs.databricks.com/en/delta-live-tables/settings.html#configure-...
- 2 kudos
- 5899 Views
- 3 replies
- 0 kudos
List granted access for a group or a user
Is there any way where I can see what access a group or a user have been given to objects (Tables, views, catalogs etc.)?I noticed that we have the following information_schema tables:catalog_privilegesroutine_privilegestable_privilegesschema_privile...
- 5899 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Henrik To grant a user the privilege to query system tables, a metastore admin or another privileged user must grant USE and SELECT permissions on the system schemas. GRANT USAGE ON CATALOG system TO <user_name>; GRANT USAGE ON SCHEMA informa...
- 0 kudos
- 19651 Views
- 3 replies
- 3 kudos
org.apache.spark.SparkException: Job aborted due to stage failure:
HiI have around 20 million records in my DF, and want to save it in HORIZINTAL SQL DB.This is error:org.apache.spark.SparkException: Job aborted due to stage failure: A shuffle map stage with indeterminate output was failed and retried. However, Spar...
- 19651 Views
- 3 replies
- 3 kudos
- 3 kudos
If there are any failures which may lead to a stage retry, but retrying the stage translates into potentially having an inconsistent result (indeterminacy) then this exception is raised. The exception is raised in newer version where the validation i...
- 3 kudos
- 2051 Views
- 0 replies
- 0 kudos
Databricks + confluent schema registry - Schema not found error
I am running a Kafka producer code on Databricks 12.2. I am testing AVRO serialization of message with help of confluent schema registry. I configured 'to_avro' function to read the schema from schema registry, but I am getting the below error> org.a...
- 2051 Views
- 0 replies
- 0 kudos
- 1380 Views
- 0 replies
- 0 kudos
Identifying the Dependency Automatically on the SQL Statements
Hi All,We have a SQL script file with lots of create, insert, and SQL statements. We need to figure out how to identify the dependencies between the statements dynamically(by using either spark/sql or using any program).For example - First, we need t...
- 1380 Views
- 0 replies
- 0 kudos
- 1785 Views
- 1 replies
- 0 kudos
Materialized view in DLT pipeline
When setting up DLT pipeline, there are 3 types of product edition, Core, Pro and Advanced. When I compare DLT Classic Core and DLT Classic Pro, the difference is that DLT Classic Pro can handle CDC. Does it means if I'm using DLT Classic Core, I hav...
- 1785 Views
- 1 replies
- 0 kudos
- 0 kudos
Also, If I'm using DLT Classic Core for DLT pipeline, the materialized view will be doing a full refresh or just update on the rows that have changes?
- 0 kudos
- 1531 Views
- 0 replies
- 0 kudos
Unzip multiple zip files in databricks
I have a zip file which in turn has multiple zip files inside it. I tried to write a code in databricks notebook to unzip all these files at once, but I ran into an error. So I started to unzip these one by one, but the code which worked in unzipping...
- 1531 Views
- 0 replies
- 0 kudos
- 2610 Views
- 4 replies
- 2 kudos
Resolved! About resuming the test - [ ref:_00D61JGc4._500Vp6jMk5:ref ]
Hello @Cert-Team I am very glad to hear from you,in response of the ticket (00481969), I have received mail for rescheduling of my test to resume. I am very thankful for your quick response in this matter.I have replied in mail about the date and tim...
- 2610 Views
- 4 replies
- 2 kudos
- 2 kudos
@Cert-Team, My issue has been clear. my test has been rescheduled to the time I have mentioned.I am very grateful for your help.Thanks and Regards@SHASHANK2
- 2 kudos
- 1511 Views
- 0 replies
- 0 kudos
Unable to unzip files recursively and copy into a different folder
I am currently trying to unzip files recursively from one folder(source folder) and copy all the unzipped files into the destination folder using databricks(pyspark). The destination path is still empty even after running this code. I tried looking f...
- 1511 Views
- 0 replies
- 0 kudos
