cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
mo91
by • New Contributor III
  • 6646 Views
  • 4 replies
  • 9 kudos

Resolved! Community edition - RestException: PERMISSION_DENIED: Model Registry is not enabled for organization 2183541758974102.

Currently running this cmmd:-model_name = "Quality"model_version = mlflow.register_model(f"runs:/{run_id}/random_forest_model", model_name)# Registering the model takes a few seconds, so add a small delaytime.sleep(15)however I get this error:-RestEx...

Data Engineering
Reply
  • 6646 Views
  • 4 replies
  • 9 kudos
View Replies
Latest Reply
Prabakar
Databricks Employee
  • 9 kudos

@Martin Olowe​ There are certain limitations with the community edition and you do not have this feature there. To use this you need to go with the commercial version of Databricks as mentioned by @Hubert Dudek​ .

  • 9 kudos
3 More Replies
harish_s
by • New Contributor II
  • 7323 Views
  • 3 replies
  • 4 kudos

Resolved! Hi, I get the following error when I enable model serving for spacy model via MLFLOW.

+ echo 'GUNICORN_CMD_ARGS=--timeout 63 --workers 4 'GUNICORN_CMD_ARGS=--timeout 63 --workers 4 + mlflow models serve --no-conda -m /tmp/tmp1a4ltdrk/spacymodelv1 -h unix:/tmp/3.sock -p12022/03/01 08:26:37 INFO mlflow.models.cli: Selected backend for f...

Data Engineering
Reply
  • 7323 Views
  • 3 replies
  • 4 kudos
View Replies
Latest Reply
Prabakar
Databricks Employee
  • 4 kudos

Hi @Harish S​ this error could happen if the backend services are not updated. Are you doing this test in a PVC environment or a standard workspace?

  • 4 kudos
2 More Replies
rsp334
by • New Contributor II
  • 2168 Views
  • 0 replies
  • 3 kudos

Databricks quickstart cloudformation error

Anyone recently encountered the following error in cloudformation stack while attempting to create a databricks quickstart workspace in AWS?[ERROR] 2022-05-17T16:25:35.920Z 6593c6c0-677c-4918-bcb2-0f5fc9a1c482 Exception: An error occurred (AccessDen...

Data Engineering
Reply
  • 2168 Views
  • 0 replies
  • 3 kudos
Doaa_Rashad
by • New Contributor III
  • 13847 Views
  • 7 replies
  • 8 kudos

Resolved! import Github repo into Databricks

I am trying to import some data from a public repo in GitHub so that to use it from my Databricks notebooks.So far I tried to connect my Databricks account with my GitHub as described here, without results though since it seems that GitHub support co...

image.png image.png
Data Engineering
Reply
  • 13847 Views
  • 7 replies
  • 8 kudos
View Replies
Latest Reply
User16753725182
Databricks Employee
  • 8 kudos

Hi @Doaa MohamedRashad​ , To access this setting, you must be an Admin.Please check if you have 'Repos' enabled in the Admin Console --> Workspace settings--> Repos. 

  • 8 kudos
6 More Replies
FRG96
by • New Contributor III
  • 28440 Views
  • 4 replies
  • 7 kudos

Resolved! How to programmatically get the Spark Job ID of a running Spark Task?

In Spark we can get the Spark Application ID inside the Task programmatically using:SparkEnv.get.blockManager.conf.getAppIdand we can get the Stage ID and Task Attempt ID of the running Task using:TaskContext.get.stageId TaskContext.get.taskAttemptId...

Data Engineering
Reply
  • 28440 Views
  • 4 replies
  • 7 kudos
View Replies
Latest Reply
FRG96
New Contributor III
  • 7 kudos

Hi @Gaurav Rupnar​ , I have Spark SQL UDFs (implemented as Scala methods) in which I want to get the details of the Spark SQL query that called the UDF, especially a unique query ID, which in SparkSQL is the Spark Job ID. That's why I wanted a way to...

  • 7 kudos
3 More Replies
Bharat105
by • New Contributor
  • 1249 Views
  • 0 replies
  • 0 kudos

Unable to complete signup

I am trying signup on databricks for my organization use . I am unable to complete as i am not receiving any mail.Please help ​

Data Engineering
Reply
  • 1249 Views
  • 0 replies
  • 0 kudos
Thom
by • New Contributor
  • 701 Views
  • 0 replies
  • 0 kudos

There seems to be missing lesson files in the repo I downloaded for the Data Engineering with Databricks course. The lesson Advanced SQL Transformati...

There seems to be missing lesson files in the repo I downloaded for the Data Engineering with Databricks course. The lesson Advanced SQL Transformations refers to files that aren't in the repo. One or two other lessons were missing as well.

Data Engineering
Reply
  • 701 Views
  • 0 replies
  • 0 kudos
NicolasJ
by • New Contributor
  • 5557 Views
  • 0 replies
  • 0 kudos

How to use Apache Sedona on Delta Live tables?

Hello,I am trying to run some geospatial transformations in Delta Live Table, using Apache Sedona.I tried defining a minimal example pipeline demonstrating the problem I encounter.First cell of my Notebook, I install apache-sedona Python package:%pip...

image
Data Engineering
Reply
  • 5557 Views
  • 0 replies
  • 0 kudos
alejandrofm
by • Valued Contributor
  • 3784 Views
  • 1 replies
  • 5 kudos

Resolved! How to set a global checkpoint for all notebooks?

I have several users doing data analysis on Databricks Spark notebooks, everything is smooth, now I want to make sure that the checkpointdir is configured on the cluster start, so every user doesn't had to set it on the Notebook (ending up in a lot o...

image
Data Engineering
Reply
  • 3784 Views
  • 1 replies
  • 5 kudos
View Replies
Latest Reply
Hubert-Dudek
Databricks MVP
  • 5 kudos

@Alejandro Martinez​ , For streaming jobs, there are, but others couldn't find them. Here are spark conf Configuration - Spark 3.2.1 Documentation (apache.org)spark.sql.streaming.checkpointLocation

  • 5 kudos
kiselevanton
by • New Contributor II
  • 1572 Views
  • 0 replies
  • 2 kudos

Community edition, sign up issue

I am studying Databricks as part of Data Science with Databricks for Data Analysts Specialization. It requires me to sign up for community edition, but it simply doesn't work.I have tried to sign up 3 times with 3 different e-mails, 2 private and 1 c...

Data Engineering
Reply
  • 1572 Views
  • 0 replies
  • 2 kudos
Anuj93
by • New Contributor III
  • 1543 Views
  • 0 replies
  • 0 kudos

Mount Point Service Principle

Is there any way to find out the service principle/Azure Active Directory applicationused to mount a container?

Data Engineering
Reply
  • 1543 Views
  • 0 replies
  • 0 kudos
JeNar
by • New Contributor II
  • 1981 Views
  • 0 replies
  • 1 kudos

Email Verification Databricks Community Edition

I was try to make account in Databricks Community Edition, After finish my registration , i wait for email verification, But until now ..i didn't received email verification. And i try to make 2nd account with my another email. But it same, i didn't ...

Data Engineering
Reply
  • 1981 Views
  • 0 replies
  • 1 kudos
Cameron_Afzal
by • New Contributor II
  • 1565 Views
  • 1 replies
  • 0 kudos

I'm unable to create an account for Databricks Community Edition. I've tried multiple email addresses and browsers across multiple attempts. I...

I'm unable to create an account for Databricks Community Edition. I've tried multiple email addresses and browsers across multiple attempts. I fill out and submit the sign-up form but never receive the email and thus can't log in. Any advice? Are the...

Data Engineering
Reply
  • 1565 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
tipu
New Contributor II
  • 0 kudos

i have try the same thing it doesn't work. can someone please help us?

  • 0 kudos
Labels