cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

abhijit007
by New Contributor
  • 3381 Views
  • 1 replies
  • 0 kudos

Unable to connect Azure kafka server with public IP from databricks notebook

Hi Team,I am unable to connect (SSH connection) from Azure Databricks Notebook to Azure Kafka server.Kafka Server and Databricks both are under same resource group and region. Also in Inbound rule the port is added in Kafka server.Please help me to r...

abhijit007_0-1704345593867.png abhijit007_1-1704346105457.png
  • 3381 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, This looks like issue with networking config. Could you please check on the routing configs, firewall routes etc to make sure destination IP to 9092 is added in the Azure console? 

  • 0 kudos
311102
by New Contributor
  • 1271 Views
  • 1 replies
  • 0 kudos

user email invitation to workspace not received

HelloSince December 2023, I cannot anymore invite users to connect to my workspace as I used to. For no reason, the users I add through my admin dashboard do not receive the invitation email and thus the link to connect to the workspace.   I tried my...

  • 1271 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you also please try to add users through account console if Identity Federation is enabled? Refer: https://docs.databricks.com/en/administration-guide/users-groups/users.html#assign-a-user-to-a-workspace-using-the-account-console

  • 0 kudos
thibault
by Contributor II
  • 11372 Views
  • 11 replies
  • 6 kudos

databricks-connect 13.1.0 limitations

Hi,Quite excited to see the new release of databricks-connect, I started writing unit tests running pyspark on a databricks cluster using databricks-connect.After some successful basic unit tests, I tested just more chained transformations on a dataf...

  • 11372 Views
  • 11 replies
  • 6 kudos
Latest Reply
jackson-nline
New Contributor III
  • 6 kudos

I doubled the `spark.connect.grpc.maxInboundMessageSize` parameter to 256mb but that didn't appear to resolve anything.

  • 6 kudos
10 More Replies
udi_azulay
by New Contributor II
  • 837 Views
  • 1 replies
  • 0 kudos

Running sql command on Single User cluster vs Shared.

Hi, when i am running the below simple code over my Unity Catalog on a Shared cluster, it works very well.But on a Single User - i am getting : Failed to acquire a SAS token for list on /__unitystorage/schemas/1bb5b053-ac96-471b-8077-8288c56c8a20/tab...

  • 837 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, Could you please refer to the limitations here: https://docs.databricks.com/en/compute/access-mode-limitations.html . Please let us know if this helps. 

  • 0 kudos
Databricks_Work
by New Contributor II
  • 2275 Views
  • 1 replies
  • 0 kudos

how to access data in one databricks in another databricks

I want to acces data in another databricks in my databricks, how to do that

  • 2275 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Hello, many thanks for your question, to be able to provide you with a more precise response we required some additional information:1. When you refer databricks in my databricks are you refering to access data that is in one workspace to another wor...

  • 0 kudos
hbs59
by New Contributor III
  • 5486 Views
  • 3 replies
  • 2 kudos

Resolved! Move multiple notebooks at the same time (programmatically)

If I want to move multiple (hundreds of) notebooks at the same time from one folder to another, what is the best way to do that? Other than going to each individual notebook and clicking "Move".Is there a way to programmatically move notebooks? Like ...

  • 5486 Views
  • 3 replies
  • 2 kudos
Latest Reply
Walter_C
Databricks Employee
  • 2 kudos

You should be redirected to the KB page, but this is the information contained: Problem How to migrate Shared folders and the notebooks Cause Shared notebooks are not migrated into new workspace by default Solution Please find the script to migrate t...

  • 2 kudos
2 More Replies
Phani1
by Valued Contributor II
  • 2473 Views
  • 1 replies
  • 2 kudos

Databricks API using the personal access token

We can access the Azure databricks API using the personal access token which is created by us manually.The objective is that client don’t want to store the personal access token which may not be secure .Do we have option to generate the token during ...

  • 2473 Views
  • 1 replies
  • 2 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 2 kudos

Hi @Phani1 ,Yes, now you can use databricks Create a user token API for create access token via automated API call.Please refer below doc - Create a user token | Token API | REST API reference | Azure Databricks

  • 2 kudos
Eldar_Dragomir
by New Contributor II
  • 4496 Views
  • 3 replies
  • 0 kudos

Databricks Volume. Not able to read a file from Scala.

I used to use dbfs with mounted directories and now I want to switch to Volumes for storing my jars and application.conf for pipelines. I see the file my application.conf in Data Explorer > Catalog > Volumes, I also see the file with dbutils.fs.ls("/...

Get Started Discussions
Databricks
Unity Catalog
  • 4496 Views
  • 3 replies
  • 0 kudos
Latest Reply
argus7057
New Contributor II
  • 0 kudos

Volumes mount are accessible using scala code only on a shared cluster. On single user mode this features is not supported yet. We use init scripts to move contents from Volumes to clusters local drive, when we need to access files from Native Scala ...

  • 0 kudos
2 More Replies
ChristianRRL
by Contributor III
  • 3646 Views
  • 2 replies
  • 1 kudos

Resolved! DLT Notebook and Pipeline Separation vs Consolidation

Super basic question. For DLT pipelines I see there's an option to add multiple "Paths". Is it generally best practice to completely separate `bronze` from `silver` notebooks? Or is it more recommended to bundle both raw `bronze` and clean `silver` d...

ChristianRRL_1-1705597040187.png
  • 3646 Views
  • 2 replies
  • 1 kudos
Latest Reply
ChristianRRL
Contributor III
  • 1 kudos

This is great! I completely missed the list view before.

  • 1 kudos
1 More Replies
Phani1
by Valued Contributor II
  • 7906 Views
  • 0 replies
  • 0 kudos

Cloudera SQL

Hi Team,Could you please help me how efficiently/quickly can convert Cloudera SQL and Hive SQL Scripts to Pyspark Script.Regards,Phanindra

  • 7906 Views
  • 0 replies
  • 0 kudos
marcusfox
by New Contributor
  • 1256 Views
  • 1 replies
  • 0 kudos

Databricks setup with Azure storage

Hi ,We have an issue with our intial setup and design.We are using a single Azure, Premium, Block blob storage account with hierarchical namespace and LRS enabled.We have three containers within it, one for each environment – Dev -Test – ProdBut the ...

  • 1256 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, could you please check https://community.databricks.com/t5/data-governance/metastore-one-per-account-region-limitation/td-p/41097  and let us know if this discussion helps? 

  • 0 kudos
chrisf_sts
by New Contributor II
  • 3139 Views
  • 0 replies
  • 0 kudos

How to handle complex json schema

I have a mounted external directory that is an s3 bucket with multiple subdirectories containing call log files in json format.  The files are irregular and complex, when i try to use spark.read.json or spark.sql (SELECT *) i get the UNABLE_TO_INFER_...

Get Started Discussions
json
pyspark
schema
  • 3139 Views
  • 0 replies
  • 0 kudos
liefeld
by New Contributor
  • 2133 Views
  • 0 replies
  • 0 kudos

Foreign catalogs aren't populated.

I've created connections to various RDS Aurora databases but always get the same problem - when creating a foreign catalog only the information_schema database is shown in Catalog Explorer.  The AI chat agent has made a few ways to specify the databa...

  • 2133 Views
  • 0 replies
  • 0 kudos
wakutgba
by New Contributor
  • 1697 Views
  • 0 replies
  • 0 kudos

https://groups.google.com/g/ibm.software.network.directory-integrator/c/9ubZOuHJob4/m/JQAdpv5qAgAJ

https://feedback.azure.com/d365community/idea/e7289e29-26b4-ee11-92bc-000d3a037f01https://docs.google.com/document/d/1gVBMFUqZaVteXs9-DbZQmH1CYHajWe3EhU_h3MV4S-s/edithttps://support.google.com/looker-studio/thread/253656391https://bemorepanda.com/en/...

  • 1697 Views
  • 0 replies
  • 0 kudos
Databricks_Work
by New Contributor II
  • 2018 Views
  • 1 replies
  • 0 kudos
  • 2018 Views
  • 1 replies
  • 0 kudos
Latest Reply
Lakshay
Databricks Employee
  • 0 kudos

Hi @Databricks_Work , Vacuum and Analzye are two separate commands that are used for optimizing the queries but they perform two different operations. Vacuum is used to clear the stale data files in your delta table. Vacuum should be run after a opti...

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels