- 12245 Views
- 4 replies
- 1 kudos
Internal error. Attach your notebook to a different cluster or restart the current cluster.
Started getting this error while running all the scripts. All the scripts were running fine before. I tried de-attaching and also restart nothing seems to work.Internal error. Attach your notebook to a different cluster or restart the current cluste...
- 12245 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @210573 (Customer)​ ,I got the same error, tried to restart and create a new cluster but the solution does not work. What I did to fix the issue: Instead of putting in function, break the code out to run line by line. I just want to see where the ...
- 1 kudos
- 11518 Views
- 1 replies
- 2 kudos
Pytest imports of sibling modules when using Databricks for VSCode
Hello all, I am following the Databrick's documentation on unit testing found here: Run tests with pytest for the Databricks extension for Visual Studio Code - Azure Databricks | Microsoft LearnHowever, when taking it a step further I get an ImportEr...
- 11518 Views
- 1 replies
- 2 kudos
- 2 kudos
HelloImport errors happen often with Pytest. To Debug this error you can add this in your "test_myfunction_test.py":import sys # printing all directories for # interpreter to search sys.pathsys.path is a built-in variable within the sys module. I...
- 2 kudos
- 5128 Views
- 4 replies
- 1 kudos
Resolved! Error inviting user to workspace "Failed to add user: A user with email ... or username ... in different cases already exist in the account"
We have 3 workspaces - 1 old version in one AWS account, 2 latest versions in another.We are PAYG full edition, not using SSO.Our admins (existing DBX users in the `admins` group) can invite new users via the Admin Console from the 1 old and 1 new wo...
- 5128 Views
- 4 replies
- 1 kudos
- 1 kudos
We are facing same issue, We are on azure. @AndyAtINX you mean if user exist in workspace with abc@gmail.com we should add the user in workspace2 with abc@gmail.com not ABC@GMAIL.COM. if this the case we tried this and its not working for us.
- 1 kudos
- 5383 Views
- 6 replies
- 0 kudos
databricks-connector: Error: Cluster MASKED is in unexpected state Pending.
Is there a way to make databricks-connector wait for cluster to be running?Details:databricks-connector==13.1.0 and the python minor version of cluster and environment are both 3.10If the cluster is not running this will start it, but any commands af...
- 5383 Views
- 6 replies
- 0 kudos
- 0 kudos
FYI for anyone that finds this: This seems to be resolved in databricks-connector 14+
- 0 kudos
- 2855 Views
- 0 replies
- 0 kudos
What is the local-ssd used for in databricks?
What is the use-case for local-ssd's in databricks clusters? I noticed some clusters have many Tb's worth and some have no local ssd's.What are the pro's and con's of changing the disk size bigger and smaller? According to the docs:> The disk cache i...
- 2855 Views
- 0 replies
- 0 kudos
- 8348 Views
- 1 replies
- 0 kudos
DUPLICATE: Missing 'DBAcademy DLT' as a Cluster Policy when creating Delta Live Tables pipeline
Good afternoon,I'm currently going through Module 4 of the Data Engineering Associate pathway, specifically lesson 4.1 - DLT UI Walkthrough. We are instructed to specify the Cluster Policy as 'DBAcademy DLT' when configuring the pipeline. However, th...
- 8348 Views
- 1 replies
- 0 kudos
- 0 kudos
The policy will be available after running the notebook /Users/<YOUR USER NAME>/Data Engineering with Databricks - v3.1.4/Includes/Workspace-Setup
- 0 kudos
- 10827 Views
- 2 replies
- 0 kudos
tracing the history of a workflow
Hi!I use Databricks in Azure and I find it inconvenient not knowing the last modified user and modified time. How can I trace the history of modified time and user details? Would it be possible to deploy the workflows into higher environments?Thanks!
- 10827 Views
- 2 replies
- 0 kudos
- 0 kudos
Sorry! I added another issue at the end without mentioning it was a new issue I encountered. I had challenges in changing the owner of a workflow when I created a workflow. I ended up seeking help from another user with admin privileges to change the...
- 0 kudos
- 11190 Views
- 2 replies
- 1 kudos
Default Catalog Name setting doesn't work
I've recently started using Unity Catalog and I'm trying to set the default catalog name to something else than the hive_metastore for some of my workspaces.According to the documentation (Update an assignment | Metastores API | REST API reference | ...
- 11190 Views
- 2 replies
- 1 kudos
- 1 kudos
I found that setting the default catalog in the workspace "Admin Settings" works for Sql warehouse, spark cluster and compute polices. Consult this documentation : https://docs.databricks.com/en/data-governance/unity-catalog/create-catalogs.html#view...
- 1 kudos
- 7444 Views
- 3 replies
- 2 kudos
TPC -DS test On databricks
If I want to run TPC-DS test on databricks what are the steps involved, do we have already daya available on databricks file system or I have to download or create from somewhere.
- 7444 Views
- 3 replies
- 2 kudos
- 2 kudos
See the spark-sql-perf repo for details on how to run benchmark tests using TPC-DS - https://github.com/databricks/spark-sql-perf
- 2 kudos
- 1344 Views
- 1 replies
- 0 kudos
Bug - data profile internal code
Hi I am not sure how to post a potential bug but I stumble upon the following issue on DBR 13.2.The same code 'sometimes' works on DBR 12.2 LTS. But if I do it on a real table, this issue always occurs.
- 1344 Views
- 1 replies
- 0 kudos
- 0 kudos
Tried reproducing the issue on DBR 13.2, but unable to. find attached the screenshot How intermittently is the issue occurring ?
- 0 kudos
- 4672 Views
- 1 replies
- 0 kudos
Resolved! Merge error in streaming case
I have a streaming case, where i stream from 2 sources: source1 and source2. I write to seperate streams to pick the data up from the landing area (step1). then i write 2 extra streams to apply some tranformations in order to give them the same schem...
- 4672 Views
- 1 replies
- 0 kudos
- 0 kudos
Solved the problem by changing the cluster settings. The whole thing works when disabling Photon Acceleration...
- 0 kudos
- 4956 Views
- 0 replies
- 0 kudos
snowflake python connector import error
```--------------------------------------------------------------------------- ImportError Traceback (most recent call last) File <command-1961894174266859>:1 ----> 1 con = snowflake.connector.connect( 2 user=USER, 3 password=SNOWSQL_PWD, 4 account=A...
- 4956 Views
- 0 replies
- 0 kudos
- 3489 Views
- 2 replies
- 2 kudos
Delta Live Tables error with Kafka SSL
We have a spark streaming job that consumes data from a Kafka topic and writes out to delta tables in Unity Catalog.Looking to refactor it to use Delta Live Tables, but it appears that it is not possible at present to have a DLT Pipeline that can acc...
- 3489 Views
- 2 replies
- 2 kudos
- 2 kudos
Indeed its already patched. you just have to configure your pipeline on the "preview" channel.
- 2 kudos
- 9831 Views
- 0 replies
- 0 kudos
Resolved! How to reinstantiate the Cloudformation template for AWS
Hi Everyone!I am new to Databricks, and had chosen to use the Cloudformation template to create my AWS Workspace. I regretfully must admit I felt creative in the process and varied the suggested stackname and that must have created errors which ended...
- 9831 Views
- 0 replies
- 0 kudos
- 3724 Views
- 0 replies
- 0 kudos
Why not enable "decommissioning" in spark?
You can enable "decommissioning" in spark, which causes it to remove work from a worker when it gets a notification from the cloud that the instance goes away (e.g. SPOT instances). This is dissabled by default, but it seems like such a no-brainer to...
- 3724 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
