cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

marcusfox
by New Contributor
  • 2131 Views
  • 1 replies
  • 0 kudos

Databricks setup with Azure storage

Hi ,We have an issue with our intial setup and design.We are using a single Azure, Premium, Block blob storage account with hierarchical namespace and LRS enabled.We have three containers within it, one for each environment – Dev -Test – ProdBut the ...

  • 2131 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, could you please check https://community.databricks.com/t5/data-governance/metastore-one-per-account-region-limitation/td-p/41097  and let us know if this discussion helps? 

  • 0 kudos
chrisf_sts
by New Contributor II
  • 6633 Views
  • 0 replies
  • 0 kudos

How to handle complex json schema

I have a mounted external directory that is an s3 bucket with multiple subdirectories containing call log files in json format.  The files are irregular and complex, when i try to use spark.read.json or spark.sql (SELECT *) i get the UNABLE_TO_INFER_...

Get Started Discussions
json
pyspark
schema
  • 6633 Views
  • 0 replies
  • 0 kudos
dvmentalmadess
by Valued Contributor
  • 13292 Views
  • 9 replies
  • 1 kudos

Resolved! Terraform databricks_storage_credential has wrong External ID

We create storage credentials using Terraform. I don't see any way to specify a given External ID (DBR Account ID) when creating the credentials via Terraform or in the web UI console. However, today when I tried creating a new set of credentials usi...

Screenshot 2023-11-28 at 6.37.50 PM.png
  • 13292 Views
  • 9 replies
  • 1 kudos
Latest Reply
Mathias_Peters
Contributor II
  • 1 kudos

I tried the proposed solution using an account provider like this provider "databricks" { account_id = "ACCOUNT_ID" host = "https://accounts.cloud.databricks.com" } for creating the storage credential. However, that did not work. I got an e...

  • 1 kudos
8 More Replies
Sujitha
by Databricks Employee
  • 3506 Views
  • 1 replies
  • 2 kudos

Built-In Governance for Your Databricks Workspace

Databricks Unity Catalog simplifies data and AI governance by providing a unified solution for organizations to securely discover, access, monitor, and collaborate on a range of data and AI assets. This includes tables, ML models, files and functions...

Screenshot 2024-01-12 at 1.50.49 PM.png Screenshot 2024-01-12 at 1.49.35 PM.png
  • 3506 Views
  • 1 replies
  • 2 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 2 kudos

Thank you for sharing this @Sujitha 

  • 2 kudos
RobsonNLPT
by Contributor III
  • 2349 Views
  • 1 replies
  • 0 kudos

Databricks SQL Identifier Variables

Hi all.Just trying to implement adb sql scripts using identifier clause but I have errors like that using an example:DECLARE mytab = 'tab1'; CREATE TABLE IDENTIFIER(mytab) (c1 INT);The feature is not supported: Temporary variables are not yet support...

Get Started Discussions
adb sql identifier
  • 2349 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@RobsonNLPT  - The feature development is still in place. Just the docs are released prior to the feature availability which is an usual process. The feature will be released on preview channel with a tentative ETA on Feb 20 as of now.  Alternatively...

  • 0 kudos
liefeld
by New Contributor
  • 2605 Views
  • 0 replies
  • 0 kudos

Foreign catalogs aren't populated.

I've created connections to various RDS Aurora databases but always get the same problem - when creating a foreign catalog only the information_schema database is shown in Catalog Explorer.  The AI chat agent has made a few ways to specify the databa...

  • 2605 Views
  • 0 replies
  • 0 kudos
wakutgba
by New Contributor
  • 2037 Views
  • 0 replies
  • 0 kudos

https://groups.google.com/g/ibm.software.network.directory-integrator/c/9ubZOuHJob4/m/JQAdpv5qAgAJ

https://feedback.azure.com/d365community/idea/e7289e29-26b4-ee11-92bc-000d3a037f01https://docs.google.com/document/d/1gVBMFUqZaVteXs9-DbZQmH1CYHajWe3EhU_h3MV4S-s/edithttps://support.google.com/looker-studio/thread/253656391https://bemorepanda.com/en/...

  • 2037 Views
  • 0 replies
  • 0 kudos
questions
by New Contributor
  • 1894 Views
  • 1 replies
  • 0 kudos

Can only connect from tableau cloud using Compute Cluster

We are trying to connect tableau cloud to databricks.  We have a serverless sql warehouse and a pro warehouse, both of those warehouses are not able to connect.Can’t connect to DatabricksDetailed Error Message There was an unknown connection error to...

  • 1894 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@questions  - It seems the current catalog is set to empty. can you please change the default catalog name to hive_metastore ?

  • 0 kudos
DB_Keith
by New Contributor
  • 1438 Views
  • 1 replies
  • 0 kudos

Data view in Side Panel

Does anyone know why I cannot see the Data view in the side panel under workspace.  I see catalog instead of data.  Is this something that has been upgraded?  

  • 1438 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@DB_Keith - Data Explorer is renamed to catalog Explorer. Please refer to the release notes.  https://learn.microsoft.com/en-us/azure/databricks/release-notes/product/2023/september#data-explorer-is-now-catalog-explorer

  • 0 kudos
hukel
by Contributor
  • 1396 Views
  • 1 replies
  • 0 kudos

Parsed Logical Plan report UnresolvedHint RANGE_JOIN

I'm new to RANGE_JOIN so this may be completely normal, but I'd like confirmation.Whenever I put a RANGE_JOIN hint in my query SELECT /*+ RANGE_JOIN(pr2, 3600) */ event.FirstIP4Record FROM SCHEMA_NAME_HERE.dnsrequest event INNER JOIN SC...

  • 1396 Views
  • 1 replies
  • 0 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

@hukel  - The query above does not have a range join, the range filter is not a join condition and it is evaluated as a regular filter.  Please refer to the criteria on range join optimization for joins.  Have a condition that can be interpreted as ...

  • 0 kudos
SamAWS
by New Contributor III
  • 2461 Views
  • 1 replies
  • 1 kudos

DataFrame vs. Spark SQL

When should I use DataFrame over Spark SQL?If DataFrame is better, then why do I need Spark SQL?

  • 2461 Views
  • 1 replies
  • 1 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

@SamAWS - when you want to query the underlying data in the dataframe by creating a view or table, Spark SQL is preferred. Same can be achieved using pyspark by retaining them as a dataframe. Spark SQL is used by wider audience and easy to develop. 

  • 1 kudos
Amarjit
by New Contributor
  • 1348 Views
  • 1 replies
  • 0 kudos

Unable to create a Unity Catalog

https://accounts.azuredatabricks.net/data/createI am unable to access the above link to start with Unity Catalog.

  • 1348 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Are you an account admin? If you access to https://accounts.azuredatabricks.net/ are you able to see the console or just the workspaces that are currently available for you?

  • 0 kudos
Prashanthkumar
by New Contributor III
  • 1908 Views
  • 1 replies
  • 0 kudos

Is it possible to view Databricks cluster metrics using REST API

I am looking for some help on getting databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API.I am trying it in postman using databricks token and with my Service Principal bear...

Prashanthkumar_0-1705104692634.png
  • 1908 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

There is currently no option available to get this metrics available through API, but is coming soon.

  • 0 kudos

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels