- 3014 Views
- 3 replies
- 2 kudos
Databricks cluster failure
do you help me with the next error?MessageCluster terminated. Reason: Azure Vm Extension FailureHelpInstance bootstrap failed.Failure message: Cloud Provider Failure. Azure VM Extension stuck on transitioning state. Please try again later.VM extensio...
- 3014 Views
- 3 replies
- 2 kudos
- 4453 Views
- 6 replies
- 10 kudos
What is the best way to take care of Drop and Rename a column in Schema evaluation.
I would need some suggestion from DataBricks Folks. As per documentation in Schema Evaluation for Drop and Rename Data is overwritten. Does it means we loose data (because I read data is not deleted but kind of staged). Is it possible to query old da...
- 4453 Views
- 6 replies
- 10 kudos
- 10 kudos
Overwritte ​option will overwritte your data. If you want to change column name then you can first alter the delta table as per your need then you can append new data as well. So both problems you can resolve
- 10 kudos
- 9691 Views
- 12 replies
- 8 kudos
Cluster terminated after 120 mins and cannot restart
Last night the cluster was working properly, but this morning the cluster was terminated automatically and cannot be restarted. Got an error message under sparkUI: Could not find data to load UI for driver 5526297689623955253 in cluster 1125-062259-i...
- 9691 Views
- 12 replies
- 8 kudos
- 6252 Views
- 5 replies
- 9 kudos
Resolved! How to write same code in different locations in the DB notebook?
The old version of the notebook had this feature, where you could Ctrl+click on different positions in a notebook cell to bring the cursor there, and type to update the code in both the positions like in JupyterLab. The newer version is awesome but s...
- 6252 Views
- 5 replies
- 9 kudos
- 9583 Views
- 2 replies
- 1 kudos
Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4, localhost, executor driver): java.lang.NullPointerException
I have uploaded a csv file which have well formatted data and I was trying to use display(questions) where questions=spark.read.option("header","true").csv("/FileStore/tables/Questions.csv")This is throwing an error as follows:SparkException: Job abo...
- 9583 Views
- 2 replies
- 1 kudos
- 1124 Views
- 0 replies
- 0 kudos
Mounting a Azure Storage Account path on Databricks
Hi,I have a Databricks instance and I mounted the Azure Storage Account. When I run the following command, the output is ExecutionError: An error occurred while calling o1168.ls.: shaded.databricks.org.apache.hadoop.fs.azure.AzureException: java.util...
- 1124 Views
- 0 replies
- 0 kudos
- 2263 Views
- 2 replies
- 0 kudos
Cannot sign in at databricks partner-academy portal
Hi thereI have used my company email to register an account for databricks learning .databricks.com a while back.Now what I need to do is create an account with partner-academy.databricks.com using my company email too.However when I register at part...
- 2263 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Muthukrishnan Balasubramanian​ I got the same issue a while back what worked for me is registering using personal account on partner academy then later I changed my email to my work email. Not sure if it's the best way to sort the issue.
- 0 kudos
- 2551 Views
- 1 replies
- 0 kudos
Resolved! zip file not able to import in workspace
Dear Team,Using the community edition when i tried to import a zip file it is always throwing some error
- 2551 Views
- 1 replies
- 0 kudos
- 0 kudos
Please refer to the error in the attachment my question is this restriction is only for community edition ? or also for premium account ?
- 0 kudos
- 1641 Views
- 1 replies
- 2 kudos
Resolved! Error in DE 4.1 - DLT UI Walkthrough (from Data Engineering with Databricks v3 course)
I am working on Data Engineering with Databricks v3 course. In notebook DE 4.1 - DLT UI Walkthrough, I countered an error in cmd 11: DA.validate_pipeline_config(pipeline_language)The error message is: AssertionError: Expected the parameter "suite" to...
- 1641 Views
- 1 replies
- 2 kudos
- 2 kudos
The DA validate function is just to check that you named the pipeline correctly, set up the correct number of workers, 0, and other configurations. The name and directory aren't crucial to the learning process. The goal is to get familiar with the ...
- 2 kudos
- 1387 Views
- 1 replies
- 1 kudos
How to remove more than 4 byte characters using pyspark in databricks?
Hi community,We have the need of removing more than 4 byte characters using pyspark in databricks since these are not supported by amazon Redshift. Does someone know how can I accomplish this?Thank you very much in advanceRegards
- 1387 Views
- 1 replies
- 1 kudos
- 1 kudos
assuming you are having a string type column in pyspark dataframe, one possible way could beidentify total number of characters for each value in column (say identify no of bytes taken by each character (say b)use substring() function to select first...
- 1 kudos
- 3599 Views
- 1 replies
- 5 kudos
How do I import a notebook from workspaces to repos?
I have a few notebooks in workspaces that I created before linking repo to my git. I have tried importing them from the repo (databricks repo). The only two options are a local file from my pc or a url. The url for a notebook does not work. Do I need...
- 3599 Views
- 1 replies
- 5 kudos
- 5 kudos
Hi @Stian Arntsen​ , when you click on the down arrow beside your notebook name (in your workspace), you will have a option called 'clone'. You can use it to clone your notebook from your workspace to repos. Hope it helps!
- 5 kudos
- 10447 Views
- 1 replies
- 1 kudos
Failed to merge incompatible data types
We are processing the josn file from the storage location on every day and it will get archived once the records are appended into the respective tables.source_location_path: "..../mon=05/day=01/fld1" , "..../mon=05/day=01/fld2" ..... "..../mon=05/d...
- 10447 Views
- 1 replies
- 1 kudos
- 1 kudos
@Hare Krishnan​ the issues highlighted can easily be handled using the .option("mergeSchema", "true") at the time of reading all the files.Sample code:spark.read.option("mergeSchema", "true").json(<file paths>, multiLine=True)The only scenario this w...
- 1 kudos
- 3430 Views
- 3 replies
- 10 kudos
Resolved! How to retrieve Auto Loader client secret from Azure Key Vault?
I'm using Auto Loader in a SQL notebook and I would like to configure file notification mode, but I don't know how to retrieve the client secret of the service principal from Azure Key Vault. Is there any example notebook somewhere? The notebook is p...
- 3430 Views
- 3 replies
- 10 kudos
- 10 kudos
Hi @Magnus Johannesson​ , you must use the Secrets utility (dbutils.secrets) in a notebook or job to read a secret.https://learn.microsoft.com/en-us/azure/databricks/dev-tools/databricks-utils#dbutils-secretsHope it helps!
- 10 kudos
- 2278 Views
- 3 replies
- 2 kudos
FutureWarning: ``databricks.feature_store.entities.feature_table.FeatureTable.keys`` is deprecated since v0.3.6
I'm getting this message with the following code:from databricks import feature_store fs = feature_store.FeatureStoreClient() fs.create_table( name='feature_store.user_login', primary_keys=['user_id'], df=df_x, description='user l...
- 2278 Views
- 3 replies
- 2 kudos
- 2 kudos
Yes, it's a nice thing to do. You can report it here: https://community.databricks.com/s/topic/0TO3f000000CnKrGAK/bug-report and if it's more urgent or blocking for you, you can also open a ticket to the help center: https://docs.databricks.com/resou...
- 2 kudos
- 481 Views
- 0 replies
- 0 kudos
patientportaltvobgynÂ
patientportaltvobgyn
- 481 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 347 | |
| 286 | |
| 252 |
