- 9548 Views
- 4 replies
- 0 kudos
Resolved! String converstion to datetimestamp format
Hello i'm converting hana sql code in databricks. we have 4 columns all in string format, start date, start time, end date, endtime..1) what expression i can use to convert values of startdate & start time from string format to datetimeformat wit...
- 9548 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello Mattconcat & to_timstamp function partially worked, values with 24 timestamp format not converted. any other approach i can think .?
- 0 kudos
- 3591 Views
- 3 replies
- 1 kudos
Resolved! Unable to create SQL Warehouse using AZURE DATABRICKS Subscription
Dear Team,I am unable to create a SQL Warehouse using AZURE Databricks subscription , below are the detailsi am able to create a cluster with single node ,but i am not able to create SQL Warehouse , i am using Cluster size - 2X Small , as part of the...
- 3591 Views
- 3 replies
- 1 kudos
- 1 kudos
In portal.azure.com, write quotas and ask for an increase. https://portal.azure.com/#view/Microsoft_Azure_Capacity/QuotaMenuBlade/~/myQuotasAdditionally, in SQL warehouse "Advanced options," you can change "Spot instance policy" from "cost-optimized"...
- 1 kudos
- 5670 Views
- 1 replies
- 3 kudos
Resolved! Generate new token error
Hello. I need to install Databricks CLI. While I am trying to generate new access token (User Settings->Generate new token) I get the following error:Could not create token with comment "cli" and lifetime (seconds) of 86400.I tried with different com...
- 5670 Views
- 1 replies
- 3 kudos
- 3 kudos
Please check in the Admin console that tokens are enabled and that you can manage them.
- 3 kudos
- 5735 Views
- 3 replies
- 5 kudos
Resolved! Delta Table - Reduce time travel storage size
Hello! I am trying to understand time travel feature. I see with "DESCRIBE HISTORY" command that all the transaction history on a specific table is recorded by version and timestamp. However, I understand that this occupies a lot of storage especiall...
- 5735 Views
- 3 replies
- 5 kudos
- 5 kudos
Thank you @Werner Stinckens​ for your reply. However I still haven't managed to delete history even after setting the below. The number of history rows remains the same when running "DESCRIBE HISTORY".SET spark.databricks.delta.retentionDurationCheck...
- 5 kudos
- 3799 Views
- 5 replies
- 1 kudos
Resolved! Databricks Associate Practice Exam -query
Dear Experts, Can anyone please let me know how option "C" is the answer to Question 31 for PracticeExam-DataEngineerAssociate. https://files.training.databricks.com/assessments/practice-exams/PracticeExam-DataEngineerAssociate.pdf?_ga=2.185796329.11...
- 3799 Views
- 5 replies
- 1 kudos
- 1 kudos
Question 17 is even worse. "A data engineer is overwriting data" vs "should simply be overwritten instead"One situation I assume is DROP and CREATE and another is INSERT INTO OVERWRITE but here both are called the same.A data engineer is overwriting ...
- 1 kudos
- 1357 Views
- 0 replies
- 0 kudos
metastore is down
I am trying to run a scala notebook, but my job just spins and says Metastore is down. Can someone help me. Thanks in advance.
- 1357 Views
- 0 replies
- 0 kudos
- 2538 Views
- 2 replies
- 7 kudos
PySpark Error in Azure Databricks
I have this error - com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILEDwhen i ran dbutils.notebook.run(.......)
- 2538 Views
- 2 replies
- 7 kudos
- 7 kudos
Hi @Godswill Mbata​ ,this looks like issue seems to be with High Concurrency cluster. Could you please confirm if you are using high concurrency cluster?Please refer: https://community.databricks.com/s/question/0D53f00001cx3ybCAA/strange-error-with-d...
- 7 kudos
- 2029 Views
- 1 replies
- 2 kudos
Resolved! Dataframe to update subset of fields in table...
I have a table that I'll update with multiple inputs (csv). Is there a simple way to update my target when the source fields won't be a 1:1 match? Another challenge I've run into is that my sources don't have a header field, though I guess I could ...
- 2029 Views
- 1 replies
- 2 kudos
- 2 kudos
Read your CSV as a dataframe and than make update using Merge (upsert).
- 2 kudos
- 7504 Views
- 3 replies
- 2 kudos
Resolved! Error when using pyflink on databricks, An error occurred while trying to connect to the Java server
Hi, right now I am trying to run a pyflink script that can connect to a kafka server. When I run that script, I got an error "An error occurred while trying to connect to the Java server 127.0.0.1:35529". Do I need to install a extra jdk for that? er...
- 7504 Views
- 3 replies
- 2 kudos
- 2 kudos
did you get Flink running on the Databricks cluster? Because that seems to be the issue here.
- 2 kudos
- 37201 Views
- 8 replies
- 4 kudos
Can't login back to Databricks
Hi,I'm trying the Databricks Premium. Got registered etc. No problem.Then I setup AWS and workspace etc. No problem.Back to Databricks and now can't login. Trying to reset password but get nothing in the email.Trying other browser also doesn't work.C...
- 37201 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Debayan Mukherjee​ could you please help me out?Is there any way I can login to my Databricks account?
- 4 kudos
- 5606 Views
- 2 replies
- 8 kudos
Resolved! Error cluster launch: Security Daemon Registration
I have created a workspace in AWS with private link. When we launch a cluster we get the following error: "Security Daemon Registration Exception: Failed to set up the spark container due to an error when registering the container to security daemon"...
- 5606 Views
- 2 replies
- 8 kudos
- 8 kudos
Thanks @Sivaprasad C S​ It's already working. The problem was that the AWS STS private link was wrongly associated with the correct subnet.
- 8 kudos
- 3096 Views
- 2 replies
- 0 kudos
Unable to sign it at customer-academy.databricks.com but can't do the same at partner-academy.databricks.com
Hi thereI have used my company email to register an account for https://customer-academy.databricks.com/learn/signin a while back.Now what I need to do is create an account with https://partner-academy.databricks.com/learn/signin using my company ema...
- 3096 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello Ajay,Thank you for reaching out. Please submit a ticket to https://help.databricks.com/s/contact-us?ReqType=training . Thank you!
- 0 kudos
- 5674 Views
- 7 replies
- 2 kudos
How do I enable new Visualization tools in my Databricks notebooks?
Databricks released new visualization toolshttps://docs.databricks.com/notebooks/visualizations/index.htmlBut I don't see them in my notebook (DBR 10.4) nor I can find out how to enable them. How do I use the new tools?
- 5674 Views
- 7 replies
- 2 kudos
- 2 kudos
@Atanu Sarkar​ , I am still not seeing this. I am running a cluster on runtime 10.4 LTS ML. The "Legacy Visualizations" are the only option available. This feature should be fully deployed, yes?
- 2 kudos
- 1495 Views
- 1 replies
- 1 kudos
DBFS Delete Bug on Community Edition
I uploaded some files in Community Edition. But when I try to delete them I get the popup that say do you want to delete, and when I select delete, the popup goes away but the file doesn't delete. Seems the the GUI is bugged. However, when I use d...
- 1495 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks for reporting. We will look into it!!
- 1 kudos
- 1208 Views
- 0 replies
- 0 kudos
Issues in Metastore Migration using Databricks Migration Tool
Hi Team,As I'm performing the Databricks workspace migration, during Metastore migration I'm facing below issue.As we found differences in the Metastore table count between Legacy and Target workspace, we checked error logs.After going through Failed...
- 1208 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1613 | |
| 771 | |
| 349 | |
| 286 | |
| 253 |
