- 6522 Views
- 2 replies
- 2 kudos
Model Serving Endpoint keeps failing with SIGKILL error
I am trying to deploy a model in the serving endpoints section, but it keeps failing after attempting to create for an hour. Here are the service logs:Container failed with: 9 +0000] [115] [INFO] Booting worker with pid: 115[2023-09-15 19:15:35 +0000...
- 6522 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello @AChang, This is a common issue when the memory requirements of your model exceed the available memory on your current compute resources. Moving to a larger compute instance with more memory can help accommodate the memory requirements of yo...
- 2 kudos
- 1667 Views
- 6 replies
- 1 kudos
Facing issue with Databricks Notification with Azure Python webhook
Hi All, Could you be able to help to resolve the issue related to Azure Databricks Notification. Trigger is not happening whenever job fails or success to webhook. I have created webhook in Azure Automation account and created Python webhook and ...
- 1667 Views
- 6 replies
- 1 kudos
- 1115 Views
- 2 replies
- 0 kudos
Azure databricks connector issue while connecting Power BI to Databricks hosted on AWS
We have setup the integration between PowerBI and Databricks (hosted on AWS) using native databricks connector. However, we required Azure databricks connector to utilize the RBAC from unity catalog. We followed all the pre-requisite mentioned here ,...
- 1115 Views
- 2 replies
- 0 kudos
- 0 kudos
OAuth is already enabled in databricks and all the pre-requisites are in place.
- 0 kudos
- 635 Views
- 1 replies
- 0 kudos
unable to create DBX account : Unable to signup. Please try again, or sign in here...
Hi,I have been trying to create an account at databricks community edition but unable to do it for the past we weeks.Can someone from the databricks support team look into it? I am getting this since past few weeks. Unable to attach the HAR file. Do ...
- 635 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @rsh48, Could you please send a mail to community@databricks.com? and share your email details. Thanks.
- 0 kudos
- 7464 Views
- 1 replies
- 0 kudos
Databricks On-Premises or in Private Cloud
Hi All,Is it possible to store/process the data on-premises or in a private cloud with Databricks? Will this choice affect costs and performance? Please advise, as the customer wants the data stored on-premises or in a private cloud for security reas...
- 7464 Views
- 1 replies
- 0 kudos
- 0 kudos
@Phani1 Databricks does not provide a product that can be directly installed and self-managed on on-premises or private cloud environments. Instead, Databricks primarily operates as a managed service on public cloud platforms such as AWS, Azure, and ...
- 0 kudos
- 799 Views
- 1 replies
- 1 kudos
Multiple metastores in Unity Catalog
Hi All,Can we have more than one meta store in a region in Unity Catalog? Having a single meta store per region helps keep metadata management organized, but customers are asking for multiple meta stores for different needs. Is it possible to have se...
- 799 Views
- 1 replies
- 1 kudos
- 1 kudos
@Phani1 When attempting to create multiple metastores in a specific region, you may encounter the following error: This region already contains a metastore. Only a single metastore per region is allowed.Databricks recommends having only one metastore...
- 1 kudos
- 675 Views
- 1 replies
- 0 kudos
Clarification on VACUUM LITE operation
Hi all, I wanted some insight and clarification on the VACUUM LITE command. VACUUM | Databricks on AWSSo I am aware that the VACUUM FULL command will deletes data files outside of the retention duration and all files in the table directory not refere...
- 675 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @dbxlearner, If you set your retention duration to 3 days for a VACUUM LITE operation, it means that the command will use the Delta transaction log to identify and remove files that are no longer referenced by any table versions within the last...
- 0 kudos
- 2368 Views
- 10 replies
- 4 kudos
Custom Tag Usage Reporting
Hi Team,Does anyone have a good SQL query that I can use for showing usage costs against custom tags for example on clusters. The Account Console usage report is good but I can only seem to query one custom tag at a time and ideally I want a dashboar...
- 2368 Views
- 10 replies
- 4 kudos
- 3306 Views
- 10 replies
- 2 kudos
Unable to login to Community Edition
Facing the below issueWe were not able to find a Community Edition workspace with this email. Please login to accounts.cloud.databricks.com to find the non-community-edition workspaces you may have access to. For help, please see Community Edition Lo...
- 3306 Views
- 10 replies
- 2 kudos
- 2 kudos
Hi All,Now neither I can sign up from my account not login. o response from them so good luck to everyone out there who's trying to learn this tool.
- 2 kudos
- 737 Views
- 1 replies
- 0 kudos
Data difference between SQL warehouse and all-purpose compute
Hey everyone,Executing the following query on my sql warehouse does not return any data:select * from acc_bolt.eod.configurationhistory where netarea = '541454827900000139';However running the same query using an all-purpose compute does return the e...
- 737 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Daan , Greetings! Can you please confirm which SQL warehouse you are using here? If you are using a serverless, then can you try to run the query with PRO/Classic warehouse? Kind Regards, Ayushi
- 0 kudos
- 8216 Views
- 4 replies
- 0 kudos
Pyspark cast error
Hi All,hive> create table UK ( a decimal(10,2)) ;hive> create table IN ( a decimal(10,5)) ;hive> create view T as select a from UK union all select a from IN ;above all statements executes successfully in Hive and return results when select statement...
- 8216 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi Nandini,Thanks for sharing the above solution. To be sure my understanding is correct, could you confirm below please ?hive> create table test.UK ( a decimal(10,2)) ;hive> create table test.IN ( a decimal(10,5)) ;hive> create view test.T as select...
- 0 kudos
- 480 Views
- 1 replies
- 0 kudos
EMR cluster pyspark scripts to databricks
Hi All,The PySpark scripts currently operating on the EMR cluster need to be migrated to Databricks. Are there any tools available that can assist in minimizing the time required for code conversion? Your suggestions would be appreciated.Regards,Phan...
- 480 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Phani1, This guide can help you: https://www.databricks.com/resources/guide/emr-databricks-migration-guide
- 0 kudos
- 747 Views
- 1 replies
- 0 kudos
Airflow jobs migration to Databricks Workflows
Hi All,We need to move our Airflow jobs over to Databricks Workflows. Are there any tools out there that can help with this migration and make the process quicker? If you have any sample code or documents that could assist, I would really appreciate ...
- 747 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Phani1, Please see this post which can help you: https://community.databricks.com/t5/data-engineering/migrating-logic-from-airflow-dags-to-databricks-workflow/td-p/104501
- 0 kudos
- 1545 Views
- 2 replies
- 3 kudos
Snowflake to Databricks migration
We are working on a proposal for our existing customer to migrate approximately 500 tables and the associated business logic from Snowflake to Databricks. The business logic is currently implemented using stored procedures, which need to be converted...
- 1545 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @tarunnagpal !!Adding to what @MariuszK said,Using an LLM to accelerate the translation process is a great approach, but if the code is proprietary, it's best to use a closed model.Implementing a validation process is crucial to ensure that the tr...
- 3 kudos
- 1766 Views
- 4 replies
- 1 kudos
Different JSON Results when Running a Job vs Running a Notebook
I have a regularly scheduled job that runs a PySpark Notebook that GETs semi-structured JSON data from an external API, loads that data into dataframes, and saves those dataframes to delta tables in Databricks. I have the schema for the JSON defined ...
- 1766 Views
- 4 replies
- 1 kudos
- 1 kudos
@Alberto_Umana Sounds good, thank you for looking into it and let me know if there's any additional information I can provide in the meantime!
- 1 kudos
