- 13799 Views
- 1 replies
- 0 kudos
Accessing TenantId via secret to connect to Azure Data Lake Storage Gen2 doesn't work
Hello,I'm following instructions in this article to connect to ADLS gen2 using Azure service principal. I can access service principal's app id and secret via Databricks key vault backed secret scope. However, this doesn't work for directory-id and I...
- 13799 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod , Thanks for the prompt reply. As per the document, the syntax is the text highlighted in red below for accessing keys from secret scope in spark config. I used the same for app id too and that works. But I if use the same syntax for ...
- 0 kudos
- 1987 Views
- 1 replies
- 0 kudos
Snowflake Data Formatting Issue
I'm loading snowflake data to delta tables in databricks, few columns in snowflake data have datatype as Number (20,7) after loading to delta table it is taking as decimal (20,7), for example, if the value is 0.0000000 in snowflake then it is showing...
- 1987 Views
- 1 replies
- 0 kudos
- 0 kudos
explicit casting seems like the way to go.First try with one column, to see if that solves your issue.If so, you can write a function that casts all decimal columns to a certain precision, something like this:def convert_decimal_precision_scale(df, p...
- 0 kudos
- 4013 Views
- 4 replies
- 1 kudos
Why is importing python code supported in Repos but not in Workspaces ?
Hi, we currently use a one repo approach which does not require a local development environment (we utilize azure dev ops and nutter for automated tests). We also have shared code accross pipelines and started with %run-sytle modularization and have ...
- 4013 Views
- 4 replies
- 1 kudos
- 1 kudos
the why is most probably because of different development tracks/teams between workspace and repos.If they will consilidate in functionality? Can't tell, only Databricks knows that; but it seems reasonable to assume the files will also be added to w...
- 1 kudos
- 2543 Views
- 3 replies
- 0 kudos
hive_metastore Access Control by different cluster type
Hello Databricks Community,I'm reaching out with a query regarding access control in the hive_metastore. I've encountered behavior that I'd like to understand better and potentially address.To illustrate the situation:I've set up three users for test...
- 2543 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Debayan, thank you for your reply. with hive_metastore, still I cannot get the level of isolation, which means that if anyone activates the Single node cluster, she/he can see all the catalog, schema, and table. However, with Unity catalog appli...
- 0 kudos
- 2326 Views
- 0 replies
- 0 kudos
Workflows pricing
Hi there,I checked the Databricks page on the pricing of Databricks Workflows ( https://www.databricks.com/product/pricing/jobs ) and have a question regarding the cost components: the pricing page only mentions compute costs (depending whether it's ...
- 2326 Views
- 0 replies
- 0 kudos
- 2562 Views
- 0 replies
- 0 kudos
Databricks community edition and cloud platform
I have been using databricks in work for a few years now and absolutely love it.I have been wanting to use it at home but I dont have a ton of money to spend.Any community members can advise a good cloud option to run databricks on that's relatively ...
- 2562 Views
- 0 replies
- 0 kudos
- 4762 Views
- 0 replies
- 0 kudos
how to resolve "Error pushing changes" Remote ref update was rejected issue
how to resolve "Error pushing changes" Remote ref update was rejected issue even after having all edit access on remote ado repo
- 4762 Views
- 0 replies
- 0 kudos
- 1350 Views
- 0 replies
- 0 kudos
Thinking of Building a new generative AI speech model for personal use
I have been using a voice cloning AI for quite some time now, it works pretty fine and that made me think if I could deploy Databricks to to build and train a machine learning model for the speech technology industry. At first I'm trying to apply it ...
- 1350 Views
- 0 replies
- 0 kudos
- 2304 Views
- 2 replies
- 0 kudos
What's LLM?
What's LLM for generative AI?
- 2304 Views
- 2 replies
- 0 kudos
- 0 kudos
LLMs are large language models like Google Bard or OpenAI or ChatGPT, etc. "It is a type of artificial intelligence (AI) algorithm that uses deep learning techniques and massively large data sets to understand, summarize, generate and predict new con...
- 0 kudos
- 9455 Views
- 6 replies
- 7 kudos
Resolved! Cannot use docker container with runtime 13.1 and above
Hi,In our setup we use custom docker images on our job clusters. The setup of these images is in line with the docker images found on the databricks github page. The custom image works fine on runtimes, including 13.0. When using anything higher that...
- 9455 Views
- 6 replies
- 7 kudos
- 7 kudos
Interestingly enough I can use the exact same docker image on 13.3LTS Beta with no issues.
- 7 kudos
- 1979 Views
- 0 replies
- 0 kudos
The Voucher code provided to me by my organization is not working. Please help ASAP.
Hello,I have been recently given a voucher code by my organization to register for the Databricks Data Engineer Associate (Version 3) exam. However, when I tried to apply the code, the page kept showing "Candidate is not associated with the promotion...
- 1979 Views
- 0 replies
- 0 kudos
- 1944 Views
- 0 replies
- 0 kudos
My exam has suspended , Need help Urgently (21/08/2023)
Hello Team,I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam.I want to file a complain...
- 1944 Views
- 0 replies
- 0 kudos
- 2485 Views
- 1 replies
- 1 kudos
Compute cluster, SQL warehouse not starting - AWS
Hey,I am using "AWSQuickstartCloudformationLambda" to create a workspace in my trial environment and on the AWS side the stack create succeeded but on the Databricks side compute cluster is not starting. The error says:Aws Authorization Failure:Failu...
- 2485 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, the error suggests there is some error with the credentials. You can check the credentials chain to see where aws is looking for the credentials. https://blog.leapp.cloud/aws-sdk-unable-to-locate-credentials.You can refer to another article: http...
- 1 kudos
- 2264 Views
- 1 replies
- 0 kudos
WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
Hiiiii,I want to scratch a text from the web page with simple code:from selenium import webdriver # Set options to make browsing easier #Function is responsible to create a driver and return that driver. def get_driver(): chrome_options = webdr...
- 2264 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, Looks like the issue is with your browser, could you please try to change the browser and confirm?Please tag @Debayan with your next comment, which will get me notified. Thanks!
- 0 kudos
- 5436 Views
- 4 replies
- 0 kudos
webscraping with Firefox driver failing - Python code
Getting below exception while running web scraping script, stuck at installing firefox from snap. Getting apparmor permission issue and tried to reinstall it and still getting the same. please check the error message section. Dependencies:Python li...
- 5436 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi, This is an issue with the dependencies of different packages. Please try as below: apt get update && apt get upgrade; apt install firefox;Please tag @Debayan with your next comment which will notify me.
- 0 kudos
