- 1225 Views
- 1 replies
- 0 kudos
How to identify Worker and Driver instance in AWS console for databricks instance?
For AWS databricks, i have configured 1 worker and 1 driver node with same node type. In the AWS console, all details are same for the two instance, as instance id only different. How to identify which instance id is for worker and which one is for d...
- 1225 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Retired_mod Thanks for your response.There is no instance pool ID while configured the cluster, then how will i able to differentiate then.Could you give any alternative way for finding the driver instance id and worker instance id in the AWS con...
- 0 kudos
- 2247 Views
- 0 replies
- 0 kudos
Usage of if else condition for data check
Hi,In a particular Workflows Job, I am trying to add some data checks in between each task by using If else statement. I used following statement in a notebook to call parameter in if else condition to check logic.{"job_id": XXXXX,"notebook_params": ...
- 2247 Views
- 0 replies
- 0 kudos
- 1979 Views
- 1 replies
- 1 kudos
Databricks Repos API Limitations
Hi, I have started using databricks recently, and I'm not able find a right solution in the documentations. i have linked multiple repos in my databricks workspace in the repos folders, and I wanted to update the repos with remote AzureDevops reposit...
- 1979 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ha2001 , Good Day! Databricks API has a limit of 10 per second for the /repos/* combined requests in the workspace. You can check the below documentation for the API limit: https://docs.databricks.com/en/resources/limits.html#:~:text=Git%20fold...
- 1 kudos
- 3378 Views
- 4 replies
- 1 kudos
Resolved! Spark streaming query stops after code exception in notebook since 14.3
Hi!I am experiencing something that I cannot find in the documentation: in databricks, using the databricks runtime 13.X, when I start a streaming query (using .start method), it creates a new query and while it is running I can execute other code in...
- 3378 Views
- 4 replies
- 1 kudos
- 1136 Views
- 1 replies
- 1 kudos
Why so many different domains and accounts?
I've lost count of how many different domains and accounts Databricks is requiring for me to use their services. Every domain is requiring its own account username, password, etc., and nothing is synced. I can't even keep track of which email addre...
- 1136 Views
- 1 replies
- 1 kudos
- 1 kudos
Pluscustomer-academy.databricks.comaccounts.cloud.databricks.comdatabricks.my.site.com
- 1 kudos
- 2066 Views
- 0 replies
- 0 kudos
How is model drift calculated when the baseline table has no timestamp column?
I try to understand how Databricks computes the model drift when the baseline table is available. What I understood from the documentation is Databricks processes both the primary and the baseline tables according to the specified granularities in th...
- 2066 Views
- 0 replies
- 0 kudos
- 695 Views
- 0 replies
- 0 kudos
Unable to create workspace using quickstart method
Hi,I created first workspace successfully using AWS quickstart method.Now I want to create one more workspace, but quickstart method is not asking IAM role and bucket name and cloudformation stack is failing.
- 695 Views
- 0 replies
- 0 kudos
- 945 Views
- 0 replies
- 0 kudos
AutoML in production
I have a workflow in Databricks and an AutoML pipeline in it.I want to deploy that pipeline in production, but I want to use the shared cluster in production, since AutoML is not compatible with the shared clusters, what can be the workaround.(Is it ...
- 945 Views
- 0 replies
- 0 kudos
- 3149 Views
- 3 replies
- 1 kudos
Resolved! Databricks Private Preview Features
Hi All, I just wanted to try the new Databricks Workflow private preview features (For Each Task).Can someone please guide me on how we can enable it in our workspace as I am having the same use case in my current project where this feature can help ...
- 3149 Views
- 3 replies
- 1 kudos
- 1 kudos
If you are using Azure Databricks just raise a support request regarding Private Preview they will enable it for you !
- 1 kudos
- 1201 Views
- 0 replies
- 0 kudos
Databricks certification Voucher code
Does anyone have a voucher code they do not intend to use and willing to share. I recently lost my job so hard time. I really want to utilize my learnings and give the exam. Thank you.
- 1201 Views
- 0 replies
- 0 kudos
- 8870 Views
- 4 replies
- 0 kudos
My notebooks running in parallel no longer include jobids
Hey Databricks - what happened to the jobids that used to be returned from parallel runs? We used them to identify which link matched the output. See attached. How are we supposed to match up the links?
- 8870 Views
- 4 replies
- 0 kudos
- 0 kudos
Hey @Databricks, @Retired_mod - waccha doing? Yesterday's "newer version of the app" that got rolled out seems to have broken the parallel runs. The ephemeral notebook is missing. The job ids are missing. What's up? Benedetta
- 0 kudos
- 1485 Views
- 0 replies
- 0 kudos
Unable to build LZO-codec
Hi Community i am try to create lzo-codec in my dbfs using:https://docs.databricks.com/en/_extras/notebooks/source/init-lzo-compressed-files.htmlbut i am facing the errorCloning into 'hadoop-lzo'... The JAVA_HOME environment variable is not defined c...
- 1485 Views
- 0 replies
- 0 kudos
- 1478 Views
- 1 replies
- 1 kudos
Looking for a promo code for Databricks certification
I went through the webinar suggested and the courses mentioned, but I have not received any voucher code for the certification. Can anyone please help. Thank you so much.
- 1478 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you. The link is broken in the response for ticketing portal. I have added a ticket with the help center. Kindly let me know if anything else is needed. Appreciate the help.
- 1 kudos
- 1589 Views
- 1 replies
- 0 kudos
LLM Chatbot With Retrieval Augmented Generation (RAG)
When executing the second block under 01-Data-Preparation-and-IndexI get the following error. Please help. AnalysisException: [RequestId=c9625879-339d-45c6-abb5-f70d724ddb47 ErrorClass=INVALID_STATE] Metastore storage root URL does not exist.Please p...
- 1589 Views
- 1 replies
- 0 kudos
- 0 kudos
I fixed this issue by providing a MANAGED LOCATION for the catalog - meant updating the _resources/00-init file as follows - spark.sql(f"CREATE CATALOG IF NOT EXISTS {catalog} MANAGED LOCATION '<location path>'")
- 0 kudos
- 3595 Views
- 3 replies
- 0 kudos
Resolved! My Global Init Script doesn't run with the new(er) LTS version 13.3. It runs great on 12.2LTS
Hey Databricks, Seems like you changed the way Global Init Scripts work. How come you changed it? My Global Init Script runs great on 12.2LTS but not on the new(er) LTS version 13.3. We don't have Unity Catalog turned on. What's up with that? Ar...
- 3595 Views
- 3 replies
- 0 kudos
- 0 kudos
Thank you ChloeBors - I tried upgrading the ubuntu version per your suggestion but got a new error "Can t open lib ODBC Driver 17 for SQL Server file not found 0 SQLDriverConnect". I tried modifying this line to 18: ACCEPT_EULA=Y apt-get install mso...
- 0 kudos
