Hi Everyone,Today I gave databricks exam for and I got 64 questions and my result was exactly 70.00%(As per databricks the pass percentage is 70 or above). but still the status was showing Failed and I couldn't get certified.Can you anyone help me on...
My workspace has a couple different types of clusters, and I'm having issues using the `dbutils` filesystem utilities when connected to a shared cluster. I'm hoping you can help me fix the configuration of the shared cluster so that I can actually us...
There are two ways to grant access to DBFS using ANY FILE:To User: GRANT SELECT ON ANY FILE TO '<user_mail_id>'To Group: GRANT SELECT ON ANY FILE TO '<group_name>'"
I'm getting an errorYou haven’t configured the CLI yet! Please configure by entering `/databricks/python_shell/scripts/db_ipykernel_launcher.py configure`My cluster is running Databricks Runtime Version 10.1I've also installed mlflow to the cluster l...
Hi Everyone,I need your help/suggestion.We are using a DLT framework for our ELT process, data is received from the Source to the RAW layer in parquet format. This raw data is loaded to the Bronze layer which acts like a history table. From the BRONZ...
Hi,I see Optimize and VACUUM are running on all tables once per day automatically.that day when we performed historical deletion, we deleted the data first, and then we just ran VACUUM with zero hour retention. After some time Optimize and VACUUM (wi...
Hello,I'am having an issue where I have :A local machine in WSL 1,Python 3.8 and 3.10OpenJDK 19.0.1 (version "build 19.0.1+10-21")Compute Instance In Azure Machine LearningPython 3.8OpenJDK 8 (version "1.8.0_392")Compute Cluster in Azure Machine Lear...
Additional information I forgot to write.Compute Instance has a User managed Identity in Azure, a Service Principal access is created in databricks with its Application ID. Same with the compute cluster, it has its own User Managed Identity that is a...
Hi,I have defined a delta table with a primary key:%sql
CREATE TABLE IF NOT EXISTS test_table_pk (
table_name STRING NOT NULL,
label STRING NOT NULL,
table_location STRING NOT NULL,
CONSTRAINT test_table_pk_col PRIMARY KEY(table_name)
...
I'm with you. But it DOES make sense because DBx databases are not application databases. DBx is not intended to be used like this. DBx databases are repositories for any ingested abstract data. To manage the ingestion is purpose-built databases ...
Hey there! Thanks a bunch for being part of our awesome community!
We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solut...
How to onnect Databricks to Oracle DAS / Autonomous Database using a cloud wallet, What are the typical steps and best practices to follow. Appreciate an example code snippet for connecting to the above data source
Followed below steps to build the connection.Unzip Oracle Wallet objects and copy them to a secure location accessible by your Databricks workspace.Collaborate with your Network team and Oracle Autonomous Instance Admins to open firewalls between yo...
Could you please inform me which specific webinar participation might grant eligibility for a certification exam voucher? Additionally, I would like to know whether this voucher would cover the full cost of the certification exam or only a partial am...
Should we convert the Python-based masking logic to SQL in databricks for implementing masking? Will the masking feature continue to work while connected to Power BI?Regards,Phanindra
We have a process that will write spark sql to a file, this process will generate thousands of spark sql files in the production environment.These files will be created in the ADLS Gen2 directory.sample spark file---val 2023_I = spark.sql("select rm....
Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...
I'm looking for materials to prepare for the Databricks Certified Professional Data Engineer exam. But I see two courses titled 'Advanced Data Engineering with Databricks' in the academy (E-VDG8QV andE-19WXD1). Which one of these courses should I be ...
My Databricks Experiments is cluttered with a whole lot of experiments. Many of them are notebooks which are showing there for some reason (even though they didn't have an MLflow run associated with it). I would like to delete the experiments, but it...
When I create an External Table in unity catalog from a flattened csv folder, it works as expected: CREATE EXTERNAL LOCATION IF NOT EXISTS raw_data
URL 'abfss://raw@storage0account0name.dfs.core.windows.net'
WITH ( STORAGE CREDENTIAL `a579a...
Thanks Kaniz,I'm using an External Location authenticated using a Managed Identity. The very same used for the non-partitioned table and many others that works pretty fine. This account has Storage Blob Contributor rights for all containers and folde...
I'm trying to access to a Databricks SQL Warehouse with python. I'm able to connect with a token on a Compute Instance on Azure Machine Learning. It's a VM with conda installed, I create an env in python 3.10.from databricks import sql as dbsql
dbsq...
The issue was that the new version of databricks-sql-connector (3.0.1) does not handle well error messages. So It gave a generic error and a timeout where it should have given me 403 and instant error message without a 900 second timeout.https://gith...
