cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

karthik_p
by Esteemed Contributor
  • 531 Views
  • 1 replies
  • 0 kudos

we have enabled compute schema in system tables, not able to see cluster and node_type tables

we have enabled system tables, under system tables we are able to see compute schema, under that we are not able to see cluster and node_type in azure databricks. do we have any limitations for above tables 

  • 531 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Certainly, karthik_p! Let’s address your query regarding the cluster and node_type tables in Azure Databricks. Cluster Tables:Cluster-related information is essential for managing and monitoring your Databricks clusters. However, there are some consi...

  • 0 kudos
Shivam_Patil
by New Contributor II
  • 1714 Views
  • 2 replies
  • 0 kudos

Databricks Certified Associate Developer for Apache Spark 3.0 - Python sample paper

I am trying to get sample papers for Databricks Certified Associate Developer for Apache Spark 3.0 - Python before I attempt my exam 

  • 1714 Views
  • 2 replies
  • 0 kudos
Latest Reply
Cert-Bricks
New Contributor III
  • 0 kudos

Hi, there, This exam has a Practice Exam, I think that's what you are looking for. 

  • 0 kudos
1 More Replies
harry546
by New Contributor III
  • 711 Views
  • 1 replies
  • 0 kudos

Unable to log metrics to table/driver using spark StreamingQueryListener

Hi Everyone,Hope everyone is enjoying their holiday time.I have spark streaming job (copy activity from location A to location B) and a Query Listener and attached to the streaming job.As mentioned over the attachments I can see numInputRows gets log...

Get Started Discussions
Streamingmetrics
StreamingQueryListener
  • 711 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @harry546, It sounds like you’re dealing with a Spark Streaming job that copies data from location A to location B, and you’ve attached a Query Listener to it.  You’re encountering issues related to writing the finaldf to a Delta table and logging...

  • 0 kudos
Phani1
by Valued Contributor
  • 575 Views
  • 1 replies
  • 0 kudos

Infrastructure reverse engineering

How can we do infrastructure reverse engineering in Databricks? Any kind of solution accelerator or tool that can scan through the environment and identify the solutions /components that are used in the Databricks environment and the complexity withi...

Get Started Discussions
reverse engineering
  • 575 Views
  • 1 replies
  • 0 kudos
Latest Reply
Phani1
Valued Contributor
  • 0 kudos

Hi Team,Could you please help us on this.Regards,Phanindra

  • 0 kudos
MiroFuoli
by New Contributor II
  • 737 Views
  • 0 replies
  • 0 kudos

Unity Catalog - Limited Options for Connection Objects

I’m currently trying to create a Foreign Catalog based on a Connection object of type SQLSERVER. This would allow me to directly access our on-premises MS SQL database from within Azure Databricks using Unity Catalog.As I’m part of a large organizati...

Get Started Discussions
Connection
Foreign Catalog
SQL warehouse
Unity Catalog
  • 737 Views
  • 0 replies
  • 0 kudos
kelvinlui1234
by New Contributor
  • 606 Views
  • 1 replies
  • 0 kudos

Can not able to connect to ADLS Gen2 using ABFSS (urgent)

Hi all,I was having an error when trying to connect to azure data lake. "SSLHandshakeExceptio" Please see the detail below.Please help!

MicrosoftTeams-image (3).png
  • 606 Views
  • 1 replies
  • 0 kudos
Latest Reply
ChishGayl
New Contributor II
  • 0 kudos

The error message you provided, "SSLHandshakeExceptio," indicates that there is a problem with the SSL handshake between your system and the Azure Data Lake. This issue can be caused by various factors, such as incorrect settings, network issues, or ...

  • 0 kudos
Stewie
by New Contributor II
  • 1037 Views
  • 1 replies
  • 2 kudos

Resolved! Databricks Advanced Data Engineering Course Factually Incorrect and Misleading

On Video 4 of the Advanced Data Engineering with Databricks course at 3:08 the presenter says 'No one else can do what we can with a single solution' . This is far from truth, Palantir foundry is miles ahead of databricks in Data Governance , Ease of...

  • 1037 Views
  • 1 replies
  • 2 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 2 kudos

Hi @Stewie , In the realm of data engineering, both Databricks and Palantir Foundry are formidable contenders, each with its own strengths and capabilities.   Let’s delve into the details:   Databricks: Platform: Databricks is a cloud-based unified d...

  • 2 kudos
Phani1
by Valued Contributor
  • 3088 Views
  • 1 replies
  • 0 kudos

Error handling best practices

Hi Team,Could you please share the best practices for error handling in Databricks for the following:                  1. Notebook level 2.Job level 3. Code level(Python) 4. streaming 5. DLT & Autoloader  Kindly suggest details around  Error handling...

  • 3088 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Phani1, Certainly! Let’s explore best practices for error handling in different contexts within Databricks:   Notebook Level: Version Control: Use version control (e.g., Git) for notebooks. Regularly commit changes to the main branch to catch err...

  • 0 kudos
AFox
by Contributor
  • 381 Views
  • 1 replies
  • 0 kudos

Databricks Community Post Editor Issues

Anyone else constantly having errors with this editor when using any of the 'features' like code sample?Can we please have a Markdown Editor or at least the ability to edit the HTML this tool creates to fix all the bugs it makes?

  • 381 Views
  • 1 replies
  • 0 kudos
Latest Reply
AFox
Contributor
  • 0 kudos

Here is a fun one: "The message body contains h d, which is not permitted in this community. Please remove this content before sending your post."Had to add the space between h and d to be able to post it.  This means code samples can't contain `ch d...

  • 0 kudos
Scott_in_Zurich
by New Contributor III
  • 1439 Views
  • 2 replies
  • 2 kudos

Resolved! Tutorial "Query Data from a Notebook": access errors

I am trying to work through Tutorial: Query data from a Notebook.Access errors are defeating my attempts. Steps to reproduce:sign up for free trial through Databricks website. The path skipped the subscription-selection step and defaulted the trial t...

  • 1439 Views
  • 2 replies
  • 2 kudos
Latest Reply
Scott_in_Zurich
New Contributor III
  • 2 kudos

Done, thanks!

  • 2 kudos
1 More Replies
Reshma_123
by New Contributor II
  • 2132 Views
  • 5 replies
  • 0 kudos

Resolved! Databricks data engineer associate Exam got suspended.

@Cert-Team I have registered for Databricks certified data engineer associate exam and I have done all biometric and other prerequisites required and launched my exam. While writing the exam, it's exited twice with some technical issue although I don...

  • 2132 Views
  • 5 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Reshma_123 , I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution. 

  • 0 kudos
4 More Replies
sg-vtc
by New Contributor III
  • 1583 Views
  • 3 replies
  • 1 kudos

Resolved! Can the same Databricks account be used for both AWS and Azure

I am testing Databricks with non-AWS S3 object storage and want to test it with Databricks on Azure and Databricks on AWS.  My Databricks account is currently using Databricks on AWS with metadata and a single node compute running. Can the same accou...

  • 1583 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @sg-vtc, I want to express my gratitude for your effort in selecting the most suitable solution. It's great to hear that your query has been successfully resolved. Thank you for your contribution. 

  • 1 kudos
2 More Replies
NC
by New Contributor III
  • 2804 Views
  • 3 replies
  • 0 kudos

Resolved! Unable to use job cluster for task in workflows

Hi,I have a workflow setup in Databricks using 12.2 LTS ML.I am trying to use a job cluster for the task but i am getting the following error: Spark Conf: ‘spark.databricks.acl.enabled’ is not allowed when choosing an access modeAs a result I have to...

  • 2804 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @NC ,  The error message you're encountering, "Spark Conf: ‘spark.databricks.acl.enabled’ is not allowed when choosing an access mode," is likely due to the job cluster's access mode not being set to "assigned" or "no isolation shared" as required...

  • 0 kudos
2 More Replies
hillel1
by New Contributor II
  • 614 Views
  • 1 replies
  • 0 kudos

run spark-sql-perf from a notebook

I am looking to run spark-sql-perf on a databricks cluster I am not sure how to create a session that add spark-sql-perf jar dependencies 

  • 614 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @hillel1, Certainly! You must add the necessary JAR dependencies to run spark-sql-perf on a Databricks cluster.    Let’s walk through the steps:   Create a Spark Session: In Databricks, you don’t need to create a Spark session explicitly. It’s aut...

  • 0 kudos
aayusha3
by New Contributor II
  • 749 Views
  • 2 replies
  • 0 kudos

Internal error: Attach your notebook to a different compute or restart the current compute.

I am currently using a personal computer cluster [13.3 LTS (includes Apache Spark 3.4.1, Scala 2.12)] on GCP attached to a notebook. After running a few command lines without an issue, I end up getting this errorInternal error. Attach your notebook t...

aayusha3_0-1700256903637.png
  • 749 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @aayusha3, Could you help us with the resource allocation for your cluster?  

  • 0 kudos
1 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors