- 3449 Views
- 2 replies
- 3 kudos
Count on External Table to Azure Data Storage is taking too long
I have created an External table to Azure Data Lake Storage Gen2.The Container has about 200K Json files.The structure of the json files are created with```CREATE EXTERNAL TABLE IF NOT EXISTS dbo.table( ComponentInfo STRUCT<ComponentHost: STRING, ...
- 3449 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @Ena Vu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 3 kudos
- 3424 Views
- 3 replies
- 1 kudos
Unable to update permissions in Unity Catalog object in Single User Mode DBR 11.2
We're trying to update permissions of catalogs in Single User Cluster Mode but running into following error We were able to update permission in Shared Mode. We used Shared mode to create objects but using single user mode to update permission seems...
- 3424 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Parth Salvi​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 1 kudos
- 5646 Views
- 4 replies
- 0 kudos
Temporarily disable Photon
Is it possible to temporarily disable Photon?I have a large workload that greatly benefits from Photon apart from a specific operation therein that is actually slowed by Photon. It's not worth creating a separate cluster for this operation however, s...
- 5646 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Aaron Morgan​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 0 kudos
- 12710 Views
- 4 replies
- 0 kudos
Resolved! String converstion to datetimestamp format
Hello i'm converting hana sql code in databricks. we have 4 columns all in string format, start date, start time, end date, endtime..1) what expression i can use to convert values of startdate & start time from string format to datetimeformat wit...
- 12710 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello Mattconcat & to_timstamp function partially worked, values with 24 timestamp format not converted. any other approach i can think .?
- 0 kudos
- 4787 Views
- 3 replies
- 1 kudos
Resolved! Unable to create SQL Warehouse using AZURE DATABRICKS Subscription
Dear Team,I am unable to create a SQL Warehouse using AZURE Databricks subscription , below are the detailsi am able to create a cluster with single node ,but i am not able to create SQL Warehouse , i am using Cluster size - 2X Small , as part of the...
- 4787 Views
- 3 replies
- 1 kudos
- 1 kudos
In portal.azure.com, write quotas and ask for an increase. https://portal.azure.com/#view/Microsoft_Azure_Capacity/QuotaMenuBlade/~/myQuotasAdditionally, in SQL warehouse "Advanced options," you can change "Spot instance policy" from "cost-optimized"...
- 1 kudos
- 7984 Views
- 1 replies
- 3 kudos
Resolved! Generate new token error
Hello. I need to install Databricks CLI. While I am trying to generate new access token (User Settings->Generate new token) I get the following error:Could not create token with comment "cli" and lifetime (seconds) of 86400.I tried with different com...
- 7984 Views
- 1 replies
- 3 kudos
- 3 kudos
Please check in the Admin console that tokens are enabled and that you can manage them.
- 3 kudos
- 8759 Views
- 3 replies
- 5 kudos
Resolved! Delta Table - Reduce time travel storage size
Hello! I am trying to understand time travel feature. I see with "DESCRIBE HISTORY" command that all the transaction history on a specific table is recorded by version and timestamp. However, I understand that this occupies a lot of storage especiall...
- 8759 Views
- 3 replies
- 5 kudos
- 5 kudos
Thank you @Werner Stinckens​ for your reply. However I still haven't managed to delete history even after setting the below. The number of history rows remains the same when running "DESCRIBE HISTORY".SET spark.databricks.delta.retentionDurationCheck...
- 5 kudos
- 4973 Views
- 5 replies
- 1 kudos
Resolved! Databricks Associate Practice Exam -query
Dear Experts, Can anyone please let me know how option "C" is the answer to Question 31 for PracticeExam-DataEngineerAssociate. https://files.training.databricks.com/assessments/practice-exams/PracticeExam-DataEngineerAssociate.pdf?_ga=2.185796329.11...
- 4973 Views
- 5 replies
- 1 kudos
- 1 kudos
Question 17 is even worse. "A data engineer is overwriting data" vs "should simply be overwritten instead"One situation I assume is DROP and CREATE and another is INSERT INTO OVERWRITE but here both are called the same.A data engineer is overwriting ...
- 1 kudos
- 1692 Views
- 0 replies
- 0 kudos
metastore is down
I am trying to run a scala notebook, but my job just spins and says Metastore is down. Can someone help me. Thanks in advance.
- 1692 Views
- 0 replies
- 0 kudos
- 3098 Views
- 2 replies
- 7 kudos
PySpark Error in Azure Databricks
I have this error - com.databricks.WorkflowException: com.databricks.NotebookExecutionException: FAILEDwhen i ran dbutils.notebook.run(.......)
- 3098 Views
- 2 replies
- 7 kudos
- 7 kudos
Hi @Godswill Mbata​ ,this looks like issue seems to be with High Concurrency cluster. Could you please confirm if you are using high concurrency cluster?Please refer: https://community.databricks.com/s/question/0D53f00001cx3ybCAA/strange-error-with-d...
- 7 kudos
- 2383 Views
- 1 replies
- 2 kudos
Resolved! Dataframe to update subset of fields in table...
I have a table that I'll update with multiple inputs (csv). Is there a simple way to update my target when the source fields won't be a 1:1 match? Another challenge I've run into is that my sources don't have a header field, though I guess I could ...
- 2383 Views
- 1 replies
- 2 kudos
- 2 kudos
Read your CSV as a dataframe and than make update using Merge (upsert).
- 2 kudos
- 8356 Views
- 3 replies
- 2 kudos
Resolved! Error when using pyflink on databricks, An error occurred while trying to connect to the Java server
Hi, right now I am trying to run a pyflink script that can connect to a kafka server. When I run that script, I got an error "An error occurred while trying to connect to the Java server 127.0.0.1:35529". Do I need to install a extra jdk for that? er...
- 8356 Views
- 3 replies
- 2 kudos
- 2 kudos
did you get Flink running on the Databricks cluster? Because that seems to be the issue here.
- 2 kudos
- 40930 Views
- 8 replies
- 4 kudos
Can't login back to Databricks
Hi,I'm trying the Databricks Premium. Got registered etc. No problem.Then I setup AWS and workspace etc. No problem.Back to Databricks and now can't login. Trying to reset password but get nothing in the email.Trying other browser also doesn't work.C...
- 40930 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi @Debayan Mukherjee​ could you please help me out?Is there any way I can login to my Databricks account?
- 4 kudos
- 6340 Views
- 2 replies
- 8 kudos
Resolved! Error cluster launch: Security Daemon Registration
I have created a workspace in AWS with private link. When we launch a cluster we get the following error: "Security Daemon Registration Exception: Failed to set up the spark container due to an error when registering the container to security daemon"...
- 6340 Views
- 2 replies
- 8 kudos
- 8 kudos
Thanks @Sivaprasad C S​ It's already working. The problem was that the AWS STS private link was wrongly associated with the correct subnet.
- 8 kudos
- 3752 Views
- 2 replies
- 0 kudos
Unable to sign it at customer-academy.databricks.com but can't do the same at partner-academy.databricks.com
Hi thereI have used my company email to register an account for https://customer-academy.databricks.com/learn/signin a while back.Now what I need to do is create an account with https://partner-academy.databricks.com/learn/signin using my company ema...
- 3752 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello Ajay,Thank you for reaching out. Please submit a ticket to https://help.databricks.com/s/contact-us?ReqType=training . Thank you!
- 0 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
