Support FAQs articles

Help Center login changes (August 3rd, 2024)

Sujitha — Wed, 14 Aug 2024 15:19:51 GMT

The purpose of this FAQ document is to provide users and partners with answers to common queries or concerns related to the Databricks Help Center Single Sign-On process.

I am a Databricks customer. Does anything change how I access Databricks platform account/workspace?

No, Databricks platform login remains the same. If you use your company’s enterprise SSO or have enabled Email OTP, please continue using that. This change (on Aug 3rd, 2024), is applicable to a few Databricks web resources (Databricks Academy, Partner Portal, and Help Center).

I am a Databricks customer who has enterprise SSO turned on to access Databricks platform. Does this release affect how I login to Databricks Platform with my enterprise account?

I am a Databricks customer and has Databricks platform credentials. Can I use the Email OTP workflow instead of the product login options?

No, the Email OTP workflow (The last option in the login screen) would work only for a few selected groups of users (non-customers). Product login options are made available so that customers can use their own enterprise SSO solution to log in to Databricks web resources (e.g. Customer Academy, Community, and Help Center) and also to enable seamless navigation from Databricks products to the portals.

I am an Azure Databricks customer. Why am I not provided the same options as AWS and GCP customers to access Help Center?

Azure Databricks support process is different from AWS or GCP Databricks support process.
Azure customer support is through Microsoft Support and we are not changing anything in that process.

Users can log in to Databricks Platform. But they see an error stating that ‘User account cannot be found or User account does not exist’ while using the same credentials to log in to Help Center.

If the customer uses enterprise SSO to log in to the Databricks platform, please contact the customer’s Databricks administrator to ensure that SSO is enabled at the account level instead of the workspace level. Instructions can be found here.

I am a Databricks customer who usually navigates to the Help Center from the Databricks workspace. Do I have to re-authenticate to access the Help Center?

If you are an AWS or GCP customer, you do not need to re-authenticate to access the Help Center.

You will be asked to re-authenticate in one of the following cases:

If you have enterprise SSO configured at the workspace level instead of the account level. Instructions to configure enterprise SSO in Databricks can be found here.
If you are an Azure Databricks customer. Azure Databricks support process is different from the AWS or GCP Databricks support process. Azure customer support is through Microsoft Support and we are not changing anything in that process.

I am a Databricks customer who logs in to Databricks using my company’s enterprise SSO. Does anything need to change in my company’s enterprise SSO configuration to allow logging into the Help Center with the same credentials?

I am a Databricks customer who uses Email OTP. Would I get 2 email OTPs?

No, typically you won’t get 2 OTPs.

If you log in to the Databricks Platform using Email OTP, choose login with AWS or GCP (One of the top 2 options on the Help Center login page) and go through the Databricks Platform Email OTP option.

If you are logging into our SSO platform for the first time using a Databricks Platform account, we validate your email by sending an Email OTP. But this is a one-time validation. You will get only one Email OTP in the subsequent logins.

I am a Microsoft employee who needs to login to Databricks Help Center. How can I login?

Microsoft employees who have access to Help Center can use the email OTP workflow to login.

Is there any change in the account creation or registration process?

There is no change to the registration/account creation process. Whenever a new support contract is signed, accounts are onboarded to Help Center by Databricks support team.

What should I (user) do if I'm still unable to log in after several attempts?

If you are using Email OTP, please ensure that you are using the correct email address and entering the correct OTP/Password. Additionally, we recommend using incognito mode while accessing the platform to avoid any cached data issues.

If you are able to log in to your Databricks account/workspace with your enterprise SSO but cannot access the Help Center, please contact your Databricks administrator to ensure that SSO is enabled at the account level instead of the workspace level. Instructions can be found here.

If you are still unable to log in after several attempts, please contact support for further assistance, as there may be an issue with your account or the authentication process.

User workflow seems to have stuck at any stage of the process

Try accessing the Help Center in an incognito window.

Ensure you are using the correct email address and entering the correct OTP/Password.

If the issue persists, please clear your browser cache and try again.

If you are still unable to log in after several attempts, please contact support for further assistance, as there may be an issue with your account or the authentication process.

Why am I not receiving the email with the OTP?

If you are not receiving the email with your OTP (One-Time Password), please follow these steps:

Verify Email Address: Ensure that the email address entered is correct and that you have access to this email account.

Check Spam/Junk Folder: Sometimes, the OTP email might be mistakenly directed to your Spam or Junk folder. Please check there.

Retry the Login Process: After verifying the above, please retry the login process to request a new OTP.

Contact Databricks Support: If you are still unable to log in after several attempts, please contact support for further assistance, as there may be an issue with your account or the authentication process.

How long is the OTP valid for?

Passwordless OTPs are valid for 5 minutes. If your OTP has expired, you will need to request a new one.

Can I use the OTP multiple times?

No, for security reasons, passwordless OTPs are intended for single use only. Each OTP can be used just once to log in. If you need to log in again, please request a new OTP.

My email OTP has expired. How can I login?

If your email OTP has expired, please request a new one and retry the login process.

Do I need to set up anything special on my device to use the passwordless login (applicable only to a select group of users)?

To use a passwordless login, you just need access to the email account associated with your Help Center profile. No additional device setup is required. You also have the option to configure passkey, which uses biometric authentication, but this is an optional step. If you have any further questions, please feel free to contact Databricks support for assistance.

What is this Passkey enrollment screen that I see when going through Email OTP?

A passkey is a digital credential that allows users to sign in to websites and apps without a password or username. Passkeys are designed to be a safer and easier alternative to passwords. Enrolling passkey is optional, but we recommend passkey enrollment to keep your account more secure. More information can be found here.

Databricks Partner Portal SSO (August 3rd, 2024)

Sujitha — Fri, 12 Jul 2024 07:45:03 GMT

The purpose of this FAQ document is to provide users and partners with answers to common queries or concerns related to the Databricks Help Center Single Sign-On process.

I am a Databricks partner. Does anything change how I access Databricks Platform account/workspace?

No, the Databricks platform login remains the same. If you use your company’s enterprise SSO or have enabled Email OTP, please continue using that. This change (on August 3rd, 2024) applies to a few Databricks web resources (Databricks Academy, Partner Portal, and Community).

Is there any change in the account creation or registration process?

There is no change to the registration/account creation process. For new partner contact additions from your organization, the current Partner Registration steps have not changed. Once new contacts submit their request to be added to the Databricks Partner Portal, Databricks will review, and approve the request. Partner contacts will receive a Welcome Email with the relevant information.

What should I (user) do if I'm still unable to log in after several attempts?

Please ensure that you are using the correct email address and entering the correct OTP/Password. Additionally, we recommend using incognito mode while accessing the platform to avoid any cached data issues.

If you are still unable to log in after several attempts, please contact partnerops@databricks.com for further assistance, as there may be an issue with your account or the authentication process.

User workflow seems to have stuck at any stage of the process

Try accessing the Partner Portal in an incognito window.

Ensure you are using the correct email address and entering the correct OTP/Password.

If the issue persists, please clear your browser cache and try again.

If steps 1 & 2 do not resolve the issue, restart your machine and try once more.

If you are still unable to log in after several attempts, please contact partnerops@databricks.com for further assistance. There may be an issue with your account or the authentication process.

Why am I not receiving the email with the OTP?

If you are not receiving the email with your OTP (One-Time Password), please follow these steps:

Verify Email Address: Ensure that the email address entered is correct and that you have access to this email account.

Check Spam/Junk Folder: Sometimes, the OTP email might be mistakenly directed to your Spam or Junk folder. Please check there.

Retry the Login Process: After verifying the above, please retry the login process to request a new OTP.

Contact Databricks Support: If you are still unable to log in after several attempts, please contact partnerops@databricks.com for further assistance. There may be an issue with your account or the authentication process.

How long is the OTP valid for?

Passwordless OTPs are valid for 5 minutes. If your OTP has expired, you will need to request a new one.

Can I use the OTP multiple times?

No, for security reasons, passwordless OTPs are intended for single use only. Each OTP can be used just once to log in. If you need to log in again, please request a new OTP.

My email OTP has expired. How can I login?

If your email OTP has expired, please request a new one and retry the login process.

Do I need to set up anything special on my device to use the passwordless login?

To use a passwordless login, you just need access to the email account associated with your Partner Portal profile. No additional device setup is required. You also have the option to configure passkey, which uses biometric authentication, but this is an optional step. If you have any further questions, please feel free to contact Databricks support for assistance.

What is this Passkey enrollment screen that I see when going through Email OTP?

Databricks Community SSO (August 3rd, 2024)

Sujitha — Tue, 30 Jul 2024 15:38:07 GMT

The purpose of this FAQ document is to provide users and partners with answers to common queries or concerns related to the Databricks Community Single Sign-On process.

Why am I being asked to update my primary and recovery email?

Primary email: This should be an active email that you have access to, to receive and use Email OTP.
Recovery email: If you lose access to your primary email in the future, your profile data can be recovered using the recovery email.

What should I (user) do if I'm still unable to log in after several attempts?

If you are still unable to log in after several attempts, please contact databricks-community@databricks.com for further assistance. There may be an issue with your account or the authentication process.

User workflow seems to have stuck at any stage of the process

Try accessing the Community in an incognito window.

Ensure you are using the correct email address and entering the correct OTP/Password.
If the issue persists, please clear your browser cache and try again.
If steps 1 & 2 do not resolve the issue, restart your machine and try once more.

If you are still unable to log in after several attempts, please contact databricks-community@databricks.com for further assistance, as there may be an issue with your account or the authentication process.

Why am I not receiving the email with the OTP?

If you are not receiving the email with your OTP (One-Time Password), please follow these steps:

Verify Email Address: Ensure that the email address entered is correct and that you have access to this email account.

Check Spam/Junk Folder: Sometimes, the OTP email might be mistakenly directed to your Spam or Junk folder. Please check there.

Retry the Login Process: After verifying the above, please retry the login process to request a new OTP.
Contact Databricks Support: If you are still unable to log in after several attempts, please contact databricks-community@databricks.com for further assistance. There may be an issue with your account or the authentication process.

How long is the OTP valid for?

Passwordless OTPs are valid for 5 minutes. If your OTP has expired, you will need to request a new one.

Can I use the OTP multiple times?

No, for security reasons, passwordless OTPs are intended for single use only. Each OTP can be used just once to log in. If you need to log in again, please request a new OTP.

My email OTP has expired. How can I login?

If your email OTP has expired, please request a new one and retry the login process.

Do I need to set up anything special on my device to use the passwordless login?

To use a passwordless login, you just need access to the email account associated with your Community profile. No additional device setup is required. You also have the option to configure passkey, which uses biometric authentication, but this is an optional step. If you have any further questions, please feel free to contact Databricks databricks-community@databricks.com for assistance.

What is this Passkey enrollment screen that I see when going through Email OTP?

A passkey is a digital credential that allows users to sign in to websites and apps without a password or username. Passkeys are designed to be a safer and easier alternative to passwords. Enrolling a passkey is optional, but we recommend passkey enrollment to keep your account more secure. More information can be found here.

Users can log in to Databricks Platform. But they see an error stating that ‘User account cannot be found or User account does not exist’ while using the same credentials to log in to Community.

I am a Databricks customer. Does anything change how I access Databricks platform account/workspace?

I am a Databricks customer who has enterprise SSO turned on to access Databricks platform. Does this release affect how I login to Databricks Platform with my enterprise account?

I am a Databricks customer and have Databricks platform credentials. Can I use the Email OTP workflow instead of the product login options?

Yes, you can log in with the Email OTP workflow (the Last option on the login screen) even if you are a customer. Product login options are made available so that customers can use their own enterprise SSO solution to log in to Databricks web resources (e.g., Customer Academy, Community, and Help Center) and enable seamless navigation from the Databricks product to the portals.

I am a Databricks customer who usually navigates to Community from the Databricks workspace. Do I have to re-authenticate to access Community?

If you are a customer, you do not need to re-authenticate to access Community.

You will be asked to re-authenticate in one of the following cases.

If you have enterprise SSO configured at the workspace level instead of the account level. Instructions to configure enterprise SSO in Databricks can be found here.
If you have a Databricks workspace session without an account-level session.

I am a Databricks customer. My existing Community profile is created with a different email address than what I use to log in to the the Databricks platform. Would I lose access to my existing profile if I log in with Databricks product credentials to Community?

We are providing an option to link your existing account to Databricks platform login when you access Community for the first time after our release on August 3rd.

Once you successfully authenticate to Community with Databricks Platform credentials, you will be taken to a screen where you can enter the email address of the existing Community profile. So, your existing profile will be ‘linked’ to your Databricks platform

I am a Databricks customer. When I try to access Community and go through the Databricks Platform login, I see a screen to enter my existing profile’s email. What is that?

This option can be used if the Databricks customer already has a Community profile, but the email associated with that profile is different from the email address associated with the Databricks platform. In that case, we give the user an option to ‘link’ their existing Community profile to the Databricks platform login.

I am a Databricks customer. My existing Community profile is created with a different email address than what I use to log in to the Databricks platform. I saw the screen to enter my existing profile email, but I skipped it. Now, a new profile has been created in the Community, and I have lost the data associated with my old profile. What should I do?

The ‘Linking’ option appears only when the user logs in through the Databricks SSO platform for the first time (Do not confuse this with Databricks product/platform/account/workspace login).

Please send your ticket to Databricks Community to do one of the following:

Re-trigger the account linking option.
Transfer data associated with the existing profile to the new profile.

I am a Databricks customer who uses Email OTP. Would I get 2 email OTPs?

No, typically you won’t get 2 OTPs.

If you log in to the Databricks Platform using Email OTP, choose login with AWS or Azure (One of the top 2 options on the Help Center login page) and go through the Databricks Platform Email OTP option.

I am getting "No Module named textdistance" errors.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:01 GMT

If you are getting No Module named textdistance errors, you need to install the textdistance library.

This can be done at the cluster level or the session level.

What is the best way to train a DeepLearning model to ensure we do not encounter out of memory (OOM) errors?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:01 GMT

You should use distributed training.

By distributing the training workload among GPUs or worker nodes, you can optimize resource utilization and reduce the likelihood of ConnectionException errors and out of memory (OOM) issues.

A good option for distributed training is Horovod, a distributed deep learning framework.

The following resources can provide guidance on how to set up and use Horovod with Databricks:

HorovodRunner: distributed deep learning with Horovod (AWS | Azure | GCP)
Distributed training (AWS | Azure | GCP)

What algorithms does AutoML support?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

AutoML supports binary/multiple classification, regression, and forecasting models.

For more details, please review the How Databricks AutoML works (AWS | Azure | GCP) documentation.

When deploying and running several jobs at the same time we get the error: REQUEST_LIMIT_EXCEEDED: Your request was rejected since cluster creation, start and upsize requests within your organization have exceeded the rate limit of 100 nodes per minute. Please retry your request later, or choose a larger node type instead.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

There is a rate limit of 100 notes per minute. To ensure you do not exceed this limit, you should make adjustments to the deployment and execution of your ML jobs.

Distribute recurring workflows evenly over the planned time period

To ensure compliance with the limit of 100 nodes per minute, it is recommended to slow down the rate of cluster creation, initiation, and upscaling requests. You can achieve this by implementing a rate-limiting mechanism in your code. For instance, you can introduce delays between each request or restrict the number of requests per minute.

Select a larger node type

As indicated by the error message, an alternative approach is to choose a larger node type instead of creating multiple smaller clusters. By opting for a larger node type, you can stay within the rate limit while still providing sufficient resources for your ML jobs.

Optimize your code

Evaluate your code to identify any areas that may be responsible for unnecessary cluster creation or inefficient resource utilization. By optimizing your code, you can potentially reduce the number of requests for cluster creation, initiation, and upscaling.

My job is failing with a "ModuleNotFoundError: No module named 'tkinter'" error during model training.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

To install the tkinter package, you can run the following shell command in a notebook:

%sh sudo apt-get install python3-tk.

To install the package automatically on every cluster start, you can add the command to a cluster-scoped init script.

When using mlflow.spark.log_model() I get a "failed to save spark model via mlflowdbfs" error.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

The trace indicates a statusCode=401 error caused by com.databricks.mlflowdbfs.MlflowHttpException.

You need to disable mlflowdbfs in the environment variable before executing log_model().

Example code:

import os
os.environ["DISABLE_MLFLOWDBFS"] = "True"

While executing the regression RECIPE with Jinja2 we get TemplateNotFound errors with recipe_dag_template.html or base.html

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

The errors occur for resources/recipe_dag_template.html with the inspect() method and for base.html with the remaining methods.

To resolve TemplateNotFound errors and ensure successful display of results while executing the regression RECIPE with Jinja2, you should run the command %pip install -r ../../requirements.txt as the first command in the notebook. This installs the necessary dependencies, which are specified in the requirements.txt file

Unable to load the xgboost model after logging via mlflow. You get a "ModuleNotFoundError: No module named 'ml'" error message.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

When using MLflow to log a model, be aware of warnings like the one below:

WARNING mlflow.utils.requirements_utils: The following packages were not found in the public PyPI package index as of 2022-12-21; if these packages are not present in the public PyPI index, you must install them manually before loading your model: {'spark-xgboost'}

This warning indicates that if certain package artifacts, such as spark-xgboost, are not available in PyPI, they won't be logged in the requirements.txt file. To recreate the model environment, these dependencies must be installed explicitly.

Here's an example:

%sh
git clone https://github.com/sllynn/spark-xgboost.git;
cd spark-xgboost;
pip install -e .

While using the model serving API we get a "BAD_REQUEST: Encountered an unexpected error while evaluating the model. Verify that the serialized input Dataframe is compatible with the model for inference” error message.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

You should review the following items:

Check the request origin:

Ensure you're making the request from the intended source. Verify if the request is originating from the expected location.

Payload mismatch:

Confirm that the payload sent in the POST request aligns with the format expected by the model. Ensure consistency between the model's input requirements and the actual payload in the request.

Serving mode UI:

If using the Serving mode UI with sample data, investigate whether the model functions correctly or throws an error. This may provide insights into potential issues related to the data or the serving environment.

Multi-model issue:

If your are using multiple models in one (multi-model), package them into a single model statically, rather than dynamically loading models from the registry. This adjustment may resolve issues associated with the error.

When deploying a serverless model, we get "Signal 9" when using small/medium clusters or "Workspace quota exhausted" when using large clusters.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

Identifying the root cause of worker termination involves analyzing signals that can provide insights into the issue. Typically, these problems are associated with memory pressure, but understanding the specific events, workload type, and workload size is crucial for decoding the underlying problem.

A Workspace quota exhausted error message indicates that the default limit for provisioned concurrency has reached the maximum value of 200. This limit is determined by the highest number of concurrent requests that can be allocated across your endpoints. If an endpoint serves a model with a large size workload, supporting 16-64 concurrent requests, the maximum provisioned concurrency for that endpoint is 64. The cumulative default limit across all endpoints is 200.

If you need to extend this default limit, contact Databricks support for further assistance.

I am running MLflow with a service principal and getting a "RestException: RESOURCE_DOES_NOT_EXIST: Node ID xxxxxxxxxxxx does not exist" error message.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

If you view a stack trace and it looks similar to the following:

RestException Traceback (most recent call last)
File <command-XXXXXXXXXXXX>:7
2 mlflow.sklearn.autolog()
...

File /databricks/python/lib/python3.9/site-packages/mlflow/tracking/fluent.py:349, in start_run(run_id, experiment_id, run_name, nested, tags, description)
345 user_specified_tags[MLFLOW_RUN_NAME] = run_name
347 resolved_tags = context_registry.resolve_tags(user_specified_tags)
--> 349 active_run_obj = client.create_run(
350 experiment_id=exp_id_for_run, tags=resolved_tags, run_name=run_name
351 )
353 _active_run_stack.append(ActiveRun(active_run_obj))
354 return _active_run_stack[-1]

You should validate that the service principal has the necessary edit permissions.

By granting edit permissions to the service principal, the job using the client.create_run method (the client is an instance of MlflowClient) can run successfully.

This edit permission ensures the necessary resources, such as the node with the specified ID, are accessible and available for the MLflow operation.

When attempting to create a serving endpoint for a custom ONNX model I get a “Container creation failed" error message.

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

A Container creation failed error message usually means there are missing dependencies.

Check the pip requirements to see if there are any missing dependencies. Adding the required dependencies should resolve the error.

How do I delete rows in the feature store?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

Feature store tables are materialized tables that use delta tables underneath.

To delete data from the feature store table, you have to run a DELETE command on the rows in the underlying delta table based on the partition column.

How can I access information offline about existing features in the feature store, like feature engineering logic?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

The Databricks feature store provides a catalog that enables data scientists to search for existing features in the offline feature store. The feature store UI offers a searchable interface, allowing you to discover features and view the code used for to create them. You can navigate to the notebook or job containing the computation logic for a specific feature. Some of this information is consolidated into Unity Catalog.

Databricks recommends enabling Unity Catalog integration to address the objective of resolving duplicate features or feature conflicts in the offline feature store. This integration allows for the utilization of data lineage, which can be accessed through the Data Explorer by going to Data -> Data Explorer -> Lineage. Additionally, the INFORMATION_SCHEMA can be used to query catalog tables and list all the column names to identify and address naming conflicts.

You can refer to the documentation for detailed examples and guides on utilizing the feature store APIs, best practices, and use cases:

How can I resolve CUDA OOM issues while performing batch inference in notebooks using ML models?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

The potential root cause could be high GPU utilization while running a live experiment. This can be validated both by using the Spark UI and by using the Nvidia -smi command.

If a single GPU is explicitly used, this might cause an overload and hence OOM issues.

To avoid OOM issues consider the following suggestions:

Experiment with smaller batch sizes
Use a larger GPU
Use a different framework such as Horovod

What is the definition of "scoring" in relation to the default scoring requests limit of 200 QPS? When batching 10 data points to be scored in a single request, is it counted as a single scoring request or as 10?

Adam_Pavlacka — Thu, 11 Jan 2024 01:00:00 GMT

It is treated as a single request when batching 10 data points. This is because the batching process involves making a single endpoint request to score multiple data points simultaneously.