cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Gopi9
by • New Contributor II
  • 2395 Views
  • 2 replies
  • 0 kudos

Need Guidance on Key Rotation Process for Storage Customer-Managed Keys in Databricks Workspace

Problem Statement: We are currently utilizing customer-managed keys for Databricks compute encryption at the workspace level. As part of our key rotation strategy, we find ourselves needing to bring down the entire compute/clusters to update storage ...

Community Platform Discussions
Reply
  • 2395 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
feiyun0112
Honored Contributor
  • 0 kudos

Maybe you can use azure key vault to store customer-managed keyshttps://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#--create-an-azure-key-vault-backed-secret-scope 

  • 0 kudos
1 More Replies
alpacas
by • New Contributor II
  • 1254 Views
  • 0 replies
  • 0 kudos

Help with getting photon usage

I want to get whether photon was used for a job or not. The api lets me get this for maybe 40% of jobs through the runtime_engine field, but the majority of jobs are unspecified. How do I get whether photon was used for those cases? The docs mention ...

Community Platform Discussions
Reply
  • 1254 Views
  • 0 replies
  • 0 kudos
tajinder123
by • New Contributor II
  • 5790 Views
  • 5 replies
  • 1 kudos

Resolved! Delta External table

Hi I am new to databricks and need some inputs.I am trying to create Delta External table in databricks using existing path which contains csv files.What i observed is below code will create EXTERNAL table but provider is CSV.------------------------...

Community Platform Discussions
Reply
  • 5790 Views
  • 5 replies
  • 1 kudos
View Replies
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

@tajinder123 - can you please modify the syntax as below to create as a delta table CREATE TABLE employee123 USING DELTA LOCATION '/path/to/existing/delta/files';  

  • 1 kudos
4 More Replies
stevieg95
by • New Contributor III
  • 2793 Views
  • 2 replies
  • 5 kudos

Getting java.util.concurrent.TimeoutException: Timed out after 15 seconds on community edition

Im using databricks communtiy edition for learning purpose and im whenever im running notebook, im getting:Exception when creating execution context: java.util.concurrent.TimeoutException: Timed out after 15 seconds databricks.I have deleted cluster ...

stevieg95_0-1711552230784.png
Community Platform Discussions
Reply
  • 2793 Views
  • 2 replies
  • 5 kudos
View Replies
Latest Reply
VenuG
New Contributor III
  • 5 kudos

Hello, Databricks Team,My students are reporting none of them are able to use DBCE, and are running into this same error when they spin up an instance with defaults (DBR 12.2 LTS). Some have reported seeing this error since last night (3/26 ET). Coul...

  • 5 kudos
1 More Replies
vieiradsousa
by • New Contributor II
  • 1882 Views
  • 1 replies
  • 0 kudos

Validating Dlt Pipeline

Whenever I try validating a pipeline that already runs productively without any issue, it throws me the following error:BAD_REQUEST: Failed to load notebook '/Repos/(...).sql'. Only SQL and Python notebooks are supported currently.

Community Platform Discussions
Reply
  • 1882 Views
  • 1 replies
  • 0 kudos
View Replies
HeijerM84
by • New Contributor III
  • 6786 Views
  • 5 replies
  • 3 kudos

Resolved! Databricks runtime 14.3 gives error scala.math.BigInt cannot be cast to java.lang.Integer

We have a cluster running on 13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12).We want to test with a different type of cluster (14.3 LTS (includes Apache Spark 3.5.0, Scala 2.12))And all of a sudden we get errors that complain about a casting a Big...

Community Platform Discussions
14.3 LTS
cast
Reply
  • 6786 Views
  • 5 replies
  • 3 kudos
View Replies
Latest Reply
jeroenvs
New Contributor III
  • 3 kudos

I have logged the issue with Microsoft last week and they confirmed it is a Databricks bug. A fix is supposedly being rolled out at the moment across Databricks regions. As anticipated, we have engaged the Databricks core team to further investigate ...

  • 3 kudos
4 More Replies
tajinder123
by • New Contributor II
  • 1332 Views
  • 0 replies
  • 0 kudos

broadcasted table reuse

in spark, table1 is small and broadcasted and joined with table 2. output is stored in df1. again, table1 is required to join with table3 and output need to be stored in df2. do it need to be broadcasted again?

Community Platform Discussions
Reply
  • 1332 Views
  • 0 replies
  • 0 kudos
chari
by • Contributor
  • 1738 Views
  • 0 replies
  • 0 kudos

how to run a group of cells in databricks ?

Hello,I was experimenting with a ML model with different parameters and check the results. However, the important part of this code is contained in a couple of cells (say cell # 12, 13 & 14). I like to proceed to the next cell only when the results a...

Community Platform Discussions
Reply
  • 1738 Views
  • 0 replies
  • 0 kudos
AravindNani
by • New Contributor
  • 2841 Views
  • 1 replies
  • 0 kudos

Unable to read data from API due to Private IP Restriction

I have data in my API Endpoint but am unable to read it using Databricks. My data is limited to my private IP address and can only be accessed over a VPN connection. I can't read data into Databricks as a result of this. I can obtain the data in VS C...

Community Platform Discussions
Reply
  • 2841 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Wojciech_BUK
Valued Contributor III
  • 0 kudos

Hi AravindNaniThis is more of infrastructure questions, you have to make sure that:1) Your databricks Workspace is provisioned in VNET Injection mode2) Your VNET is either peered to "HUB" network where you have S2S VPN Connection to API or you have t...

  • 0 kudos
arkiboys
by • Contributor
  • 4362 Views
  • 1 replies
  • 0 kudos

databricks email notification

In databricks, if a job fails, then an email is sent off as notification.The recipient, receives the email with the link to the databricks workspace.Question:How is it possible the email is sent without any link, just the plain text in the email is w...

Community Platform Discussions
Reply
  • 4362 Views
  • 1 replies
  • 0 kudos
View Replies
RozaZaharieva
by • New Contributor
  • 1726 Views
  • 0 replies
  • 0 kudos

set up Azure Databricks workspace and Unity catalog - how to automate not using Terraform

Hi everyone, I am looking for a way to automate initial setup of Azure Databricks workspace and Unity Catalog but can't find anything on this topic other than Terraform. Can you share if this is possible with powershell, for example. Thank you un adv...

Community Platform Discussions
Reply
  • 1726 Views
  • 0 replies
  • 0 kudos
Shravanshibu
by • New Contributor III
  • 1082 Views
  • 0 replies
  • 0 kudos

Public preview API not working - artifact-allowlists

 I am trying to hit /api/2.1/unity-catalog/artifact-allowlists/as a part of INIT migration script. Its is in public preview, do we need to enable anything else to use a API which is in Public preview. I am getting 404 error. But using same token for ...

Community Platform Discussions
initscripts
Reply
  • 1082 Views
  • 0 replies
  • 0 kudos
SaiNeelakantam
by • New Contributor
  • 2344 Views
  • 1 replies
  • 0 kudos

How to enable "Create Vector Search Index" button in DB workspace?

How to enable "Create Vector Search Index" button in DB workspace?Following is the screenshot from the Microsoft Ignite 2023 Databricks presentation:

Community Platform Discussions
Reply
  • 2344 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
PL_db
Databricks Employee
  • 0 kudos

The feature is in public preview only in some regions, you can check the available regions in the documentation here.  In addition there are certain requirements, such as a UC enabled workspace and Serverless Compute enabled, you can check all requir...

  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Top Kudoed Authors
View all
Top