- 2165 Views
- 1 replies
- 1 kudos
Failed to create Cluster on GCP
I am getting following error while trying to create a Cluster for my Workspace Cluster creation failed: Constraint constraints/compute.disableSerialPortLogging violated for projectCloud ENV is GCP and we can't turn off the constraint mentioned above....
- 2165 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, Haven't found any solution so far. What I hoping for is to create a cluster in a way it doesn't require SerialPortLogging so that the Policy Constraint we have e.g. disableSerialPortLogging doesn't come in the way. Not sure how can we do that. Ma...
- 1 kudos
- 8651 Views
- 0 replies
- 0 kudos
Cloudera SQL
Hi Team,Could you please help me how efficiently/quickly can convert Cloudera SQL and Hive SQL Scripts to Pyspark Script.Regards,Phanindra
- 8651 Views
- 0 replies
- 0 kudos
- 2610 Views
- 1 replies
- 0 kudos
Databricks setup with Azure storage
Hi ,We have an issue with our intial setup and design.We are using a single Azure, Premium, Block blob storage account with hierarchical namespace and LRS enabled.We have three containers within it, one for each environment – Dev -Test – ProdBut the ...
- 2610 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, could you please check https://community.databricks.com/t5/data-governance/metastore-one-per-account-region-limitation/td-p/41097 and let us know if this discussion helps?
- 0 kudos
- 6939 Views
- 0 replies
- 0 kudos
How to handle complex json schema
I have a mounted external directory that is an s3 bucket with multiple subdirectories containing call log files in json format. The files are irregular and complex, when i try to use spark.read.json or spark.sql (SELECT *) i get the UNABLE_TO_INFER_...
- 6939 Views
- 0 replies
- 0 kudos
- 16618 Views
- 9 replies
- 1 kudos
Resolved! Terraform databricks_storage_credential has wrong External ID
We create storage credentials using Terraform. I don't see any way to specify a given External ID (DBR Account ID) when creating the credentials via Terraform or in the web UI console. However, today when I tried creating a new set of credentials usi...
- 16618 Views
- 9 replies
- 1 kudos
- 1 kudos
I tried the proposed solution using an account provider like this provider "databricks" { account_id = "ACCOUNT_ID" host = "https://accounts.cloud.databricks.com" } for creating the storage credential. However, that did not work. I got an e...
- 1 kudos
- 4298 Views
- 1 replies
- 2 kudos
Built-In Governance for Your Databricks Workspace
Databricks Unity Catalog simplifies data and AI governance by providing a unified solution for organizations to securely discover, access, monitor, and collaborate on a range of data and AI assets. This includes tables, ML models, files and functions...
- 4298 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for sharing this @Sujitha
- 2 kudos
- 2937 Views
- 1 replies
- 1 kudos
DLT Medallion Incremental Ingestion Pattern Approach
Hi there, I have a question regarding what would be the "recommended" incremental ingestion approach using DLT to pull raw landing data into bronze and then silver? The original approach I've been considering is to have raw CSV files arrive in a land...
- 2937 Views
- 1 replies
- 1 kudos
- 2839 Views
- 1 replies
- 0 kudos
Databricks SQL Identifier Variables
Hi all.Just trying to implement adb sql scripts using identifier clause but I have errors like that using an example:DECLARE mytab = 'tab1'; CREATE TABLE IDENTIFIER(mytab) (c1 INT);The feature is not supported: Temporary variables are not yet support...
- 2839 Views
- 1 replies
- 0 kudos
- 0 kudos
@RobsonNLPT - The feature development is still in place. Just the docs are released prior to the feature availability which is an usual process. The feature will be released on preview channel with a tentative ETA on Feb 20 as of now. Alternatively...
- 0 kudos
- 1941 Views
- 0 replies
- 0 kudos
databricks data engineer associate exam I missed retake exam
Hi Team,I had a written exam on Jan 2nd, 2024 but I failed the exam with 65% I misunderstood I needed to take the exam after 14 days.I have missed the chance. Could you please give me a chance to write exam.email:tudururavikiran@gmail.comThanks & Reg...
- 1941 Views
- 0 replies
- 0 kudos
- 2833 Views
- 0 replies
- 0 kudos
Foreign catalogs aren't populated.
I've created connections to various RDS Aurora databases but always get the same problem - when creating a foreign catalog only the information_schema database is shown in Catalog Explorer. The AI chat agent has made a few ways to specify the databa...
- 2833 Views
- 0 replies
- 0 kudos
- 2181 Views
- 0 replies
- 0 kudos
https://groups.google.com/g/ibm.software.network.directory-integrator/c/9ubZOuHJob4/m/JQAdpv5qAgAJ
https://feedback.azure.com/d365community/idea/e7289e29-26b4-ee11-92bc-000d3a037f01https://docs.google.com/document/d/1gVBMFUqZaVteXs9-DbZQmH1CYHajWe3EhU_h3MV4S-s/edithttps://support.google.com/looker-studio/thread/253656391https://bemorepanda.com/en/...
- 2181 Views
- 0 replies
- 0 kudos
- 2465 Views
- 1 replies
- 0 kudos
Can only connect from tableau cloud using Compute Cluster
We are trying to connect tableau cloud to databricks. We have a serverless sql warehouse and a pro warehouse, both of those warehouses are not able to connect.Can’t connect to DatabricksDetailed Error Message There was an unknown connection error to...
- 2465 Views
- 1 replies
- 0 kudos
- 0 kudos
@questions - It seems the current catalog is set to empty. can you please change the default catalog name to hive_metastore ?
- 0 kudos
- 1903 Views
- 1 replies
- 0 kudos
Data view in Side Panel
Does anyone know why I cannot see the Data view in the side panel under workspace. I see catalog instead of data. Is this something that has been upgraded?
- 1903 Views
- 1 replies
- 0 kudos
- 0 kudos
@DB_Keith - Data Explorer is renamed to catalog Explorer. Please refer to the release notes. https://learn.microsoft.com/en-us/azure/databricks/release-notes/product/2023/september#data-explorer-is-now-catalog-explorer
- 0 kudos
- 1977 Views
- 1 replies
- 0 kudos
Parsed Logical Plan report UnresolvedHint RANGE_JOIN
I'm new to RANGE_JOIN so this may be completely normal, but I'd like confirmation.Whenever I put a RANGE_JOIN hint in my query SELECT /*+ RANGE_JOIN(pr2, 3600) */ event.FirstIP4Record FROM SCHEMA_NAME_HERE.dnsrequest event INNER JOIN SC...
- 1977 Views
- 1 replies
- 0 kudos
- 0 kudos
@hukel - The query above does not have a range join, the range filter is not a join condition and it is evaluated as a regular filter. Please refer to the criteria on range join optimization for joins. Have a condition that can be interpreted as ...
- 0 kudos
- 3113 Views
- 1 replies
- 1 kudos
DataFrame vs. Spark SQL
When should I use DataFrame over Spark SQL?If DataFrame is better, then why do I need Spark SQL?
- 3113 Views
- 1 replies
- 1 kudos
- 1 kudos
@SamAWS - when you want to query the underlying data in the dataframe by creating a view or table, Spark SQL is preferred. Same can be achieved using pyspark by retaining them as a dataframe. Spark SQL is used by wider audience and easy to develop.
- 1 kudos
