- 26304 Views
- 7 replies
- 0 kudos
connect to azure sql database from databricks using service principal
Hi All, Can someone please help me with the Python code to connect Azure SQL Database to Databricks using Service Principle instead of directly passing username and password. I'm using above code but getting above error. Refer Screenshot 2.Please hel...
- 26304 Views
- 7 replies
- 0 kudos
- 0 kudos
First, you need to create a service principal in Azure and grant it the necessary permissions to access your Azure SQL Database to do crm data enrichment. You can do this using the Azure CLI or the Azure Portal. Ensure that your Databricks cluster ha...
- 0 kudos
- 5298 Views
- 3 replies
- 0 kudos
Requesting an exam reattempt
Hi DB SupportI gave Databricks Certified Associate Engineering Exam today but missed by just by one percent. I got 68.88% and pass is 70%.I am planning to reattempt this exam in coming days and was hoping you could help, Could you kindly give me anot...
- 5298 Views
- 3 replies
- 0 kudos
- 0 kudos
I recommend reaching out to Databricks directly or checking their official certification website for information on retake policies, voucher availability, and any discounts or promotions they may offer for reattempts.
- 0 kudos
- 6021 Views
- 0 replies
- 0 kudos
Installing fpp3 R package on Databricks
In R notebook I am running: install.packages('fpp3', dependencies = TRUE) And getting back errors: ERROR: dependency ‘vctrs’ is not available for package ‘slider’I then install 'vctrs' and it again generates similar error that some package is...
- 6021 Views
- 0 replies
- 0 kudos
- 1806 Views
- 0 replies
- 0 kudos
How do I get AWS costs from my SQL Warehouses?
Hi,How do I find the AWS associated costs from my databricks SQL warehouse usage? I tried using tags but they didn't show up in the AWS cost explorer.My use case is I am running some DBT - Databricks jobs and I want to find the cost for certain jobs....
- 1806 Views
- 0 replies
- 0 kudos
- 2820 Views
- 1 replies
- 0 kudos
Using Python UDF in Delta live table
Hello,I tried running a python UDF in a Delta Live Table workflow in Advanced mode but it did not run and gave the "Python UDF is not supported in your environment" error.Can I get a clear picture if the Python External UDFs are supported or not?
- 2820 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod I ran this SQL query in my Catalog (I'm using Unity Catalog) :CREATE OR REPLACE FUNCTION cat_projint_dev.silver.GetEditor(prompt STRING)RETURNS STRINGLANGUAGE PYTHONAS $$print(prompt)$$ Then I ran a Delta Live Table workflow using Uni...
- 0 kudos
- 7490 Views
- 1 replies
- 0 kudos
Unexpected performance behaviors due to changes in the Spark engine or Databricks runtime
Hi!We have recently upgraded our cluster from Databricks Runtime 10.4 LTS which includes Apache Spark 3.2.1 to to Databricks Runtime 13.3 LTSincludes Apache Spark 3.2.1 powered by Apache Spark 3.3.0 and noticed that one of our jobs runtime has dramat...
- 7490 Views
- 1 replies
- 0 kudos
- 0 kudos
Seems that broadcasting the smaller table in crossjoin did the magic.
- 0 kudos
- 9448 Views
- 10 replies
- 7 kudos
Facing issue in the below mentioned code: dbutils.notebook.entry_point.getDbutils().notebook().getContext().tags().get('browserHostName').get() while executing through workflow, but it's totally working fine with workspace.
Please do let me know how to resolve this issue?
- 9448 Views
- 10 replies
- 7 kudos
- 2423 Views
- 1 replies
- 0 kudos
Noisy error in Databricks.
Hello all, I need some help figuring out the exact cause of this error. While in the middle of a complex scala transformation notebook in Azure Databricks, from time to time i get an error which looks like this, can someone help?Job aborted due to st...
- 2423 Views
- 1 replies
- 0 kudos
- 11943 Views
- 0 replies
- 1 kudos
Azure Service Bus
Hi Team,My client has Azure Service Bus and wants to do streaming using DLT. In my previous client we use Event Hub to get the data. Not sure if we can do this with Service Bus.Thanks in advance,G
- 11943 Views
- 0 replies
- 1 kudos
- 17095 Views
- 1 replies
- 2 kudos
temp tables in Databricks
Hey Team,We are in a migration phase from SQL server to Databricks.In our sql server databricks, lot of times, #temp tables created and dropped like typical Datawarehousing sql.In Databricks, we can do 2 things. I want to check which one is best appr...
- 17095 Views
- 1 replies
- 2 kudos
- 2 kudos
Just to add a bit more from experienceIf you are doing to use the temp tables too many times, then using temp views is going to slow down your cluster . It is due to lazy evaluation unless you persist that view (dataframe cache or globaltempview).Eve...
- 2 kudos
- 2981 Views
- 2 replies
- 0 kudos
Error while installing SAT tool in azure databricks
Hi, was running the security analysis initializer notebook, am I'm getting an error in cmd 8 for import_dashboard_template notebook. That is notebook no.5. First got an error for cmd 8 for this line "workspacedf = spark.sql("select * from `global_te...
- 2981 Views
- 2 replies
- 0 kudos
- 0 kudos
Hii,You can double-check the structure and contents of the json_ object to ensure it contains the expected keys and values. If you still encounter issues, verify the data types and perform necessary type conversions to ensure compatibility.
- 0 kudos
- 1307 Views
- 1 replies
- 0 kudos
Data Engineer Associate V3 exam got suspended
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 15 minutes. I had also shown my exam room to the proctor. My exam got suspended due to eye movement. I was not moving my eyes away from laptop screen. Some questions ar...
- 1307 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @rahulverma22 , Please refer to this post for guidance on how to proceed: https://community.databricks.com/t5/certifications/suspended-exam-don-t-worry-our-support-team-can-help/m-p/44469#M352
- 0 kudos
- 10377 Views
- 4 replies
- 0 kudos
How to handle multilines coming from CSV file in a quoted string
How to handle multilines coming from CSV file in a quoted string
- 10377 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi, Can you try escape parameter & quote parameter to indicate which characters need to be ignored. The escape character within the quotes will be ignored. you can specify the newline character, that it needs to be ignored. please refer to below doc...
- 0 kudos
- 21496 Views
- 3 replies
- 1 kudos
Resolved! Databricks Python Import Bug
Hello,I am reaching to the community in order to shed some light into a Bug I have been encountering recently. The two setups are as follows:SETUP-1 (WORKS):Python Notebook in Workspace FileSystem (this is Shared/folder/...)Python custom wheel librar...
- 21496 Views
- 3 replies
- 1 kudos
- 1 kudos
Solution has comed with an update, as stated in Work with Python and R modules | Databricks on AWS, In Databricks Runtime 13.0 and above, directories added to the Python sys.path are automatically distributed to all executors in the cluster. In Datab...
- 1 kudos
- 171850 Views
- 34 replies
- 6 kudos
How to read excel file using databricks
0 I have a excel file as source file and i want to read data from excel file and convert data in data frame using databricks. I have already added maven dependence for Excel file format. when i a tring below code it is giving error .(Error: java.io....
- 171850 Views
- 34 replies
- 6 kudos
- 6 kudos
To read an Excel file using Databricks, you can use the Databricks runtime, which supports multiple programming languages such as Python, Scala, and R. Here are the general steps to read an Excel file in Databricks using Python:1. **Upload the Excel ...
- 6 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 554 | |
| 349 | |
| 287 |
