cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

cony2025
by New Contributor II
  • 2611 Views
  • 4 replies
  • 0 kudos

Use Python notebook to read data from Databricks

I'm very new to Databricks. I hope this is the right place to ask this question.I want to use PySpark in a notebook to read data from a Databricks database with the below codes. databricks_host = "adb-xxxx.azuredatabricks.net" http_path = "/sql/1.0/w...

Screenshot 2025-03-04 225915.png Screenshot 2025-03-04 211512.png
  • 2611 Views
  • 4 replies
  • 0 kudos
Latest Reply
dna1
New Contributor II
  • 0 kudos

I would try changing the query to something like the following, it should return the column names in the table so you can see if the jdbc call is actually returning the data correctlySELECT * FROM wphub_poc.gold.v_d_building limit 10 

  • 0 kudos
3 More Replies
robbdunlap
by New Contributor III
  • 4278 Views
  • 8 replies
  • 22 kudos

Turn Off Auto-reveal of Navigation Sidebar

I work with the navigation sidebar closed and use the stacked hamburgers symbol in the upper left to reveal it when I want. Now, if you mouse over the left edge of the browser window too slowly it will auto-reveal the navigation sidebar. I do not wan...

  • 4278 Views
  • 8 replies
  • 22 kudos
Latest Reply
Advika
Databricks Employee
  • 22 kudos

I've checked with the team, and there's no way to turn this off. However, they are making adjustments to improve the experience, and a fix to refine the sidebar functionality is on the way.

  • 22 kudos
7 More Replies
rgcabusas
by New Contributor II
  • 2878 Views
  • 3 replies
  • 0 kudos

Virtual Learning Festival Enrollment

Hi everyone,I tried to enroll to Virtual Learning Festival: 9 April - 30 April but upon clicking the Customers & Prospects link for LEARNING PATHWAY 1: ASSOCIATE DATA ENGINEERING I got the error (refer attached image).Thank you in advance for the hel...

  • 2878 Views
  • 3 replies
  • 0 kudos
Latest Reply
rgcabusas
New Contributor II
  • 0 kudos

Hi @Advika,Refer attached image. I thought it was attached in my question. I really new to here.

  • 0 kudos
2 More Replies
Master_DataBric
by New Contributor II
  • 299 Views
  • 1 replies
  • 0 kudos

Expectation in DLT using multiple columns

Is it possible to define an  expectation  in DLT pipeline using multiple columns?For example, my source has two fields - Division, Material_Number. For division 20, material number starts with 5; for 30 material number starts with 9.Can we have this ...

  • 299 Views
  • 1 replies
  • 0 kudos
Latest Reply
ashraf1395
Honored Contributor
  • 0 kudos

Hi @Master_DataBric , Yes its possibleHere is the doc link : - https://docs.databricks.com/aws/en/dlt/expectations?language=Python- https://docs.databricks.com/aws/en/dlt/expectations?language=SQL

  • 0 kudos
TravisBrowne
by New Contributor II
  • 4834 Views
  • 2 replies
  • 1 kudos

POC Comparison: Databricks vs AWS EMR

Hello,I need some assistance with a comparison between Databricks and AWS EMR. We've been evaluating the Databricks Data Intelligence platform for a client and found it to be significantly more expensive than AWS EMR. I understand the challenge in ma...

  • 4834 Views
  • 2 replies
  • 1 kudos
Latest Reply
sandeepmankikar
Contributor
  • 1 kudos

Databricks is highly optimized for Delta, which leverages columnar storage, indexing, and caching for better performance.Instead of directly processing CSV files, convert them to Delta first, then perform aggregations and joins, see if this helps

  • 1 kudos
1 More Replies
397973
by New Contributor III
  • 383 Views
  • 1 replies
  • 1 kudos

Is it possible to concatenate two notebooks?

I don't think it's possible but I thought I would check. I need to combine notebooks. While developing I might have code in various notebooks. I read them in with "%run".Then when all looks good I combine many cells into fewer notebooks. Is there any...

  • 383 Views
  • 1 replies
  • 1 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 1 kudos

Hi @397973, Combining multiple notebooks into a single notebook isn't an out-of-the-box feature, but will try to combine %run commands ando output them to see if it works, sort of like: %run "/path/to/notebook1"%run "/path/to/notebook2"

  • 1 kudos
amit_jbs
by New Contributor II
  • 2604 Views
  • 5 replies
  • 2 kudos

In databricks deployment .py files getting converted to notebooks

A critical issue has arisen that is impacting our deployment planning for our client. We have encountered a challenge with our Azure CI/CD pipeline integration, specifically concerning the deployment of Python files (.py). Despite our best efforts, w...

  • 2604 Views
  • 5 replies
  • 2 kudos
Latest Reply
AGivenUser
New Contributor II
  • 2 kudos

Another option is Databricks Asset Bundles.

  • 2 kudos
4 More Replies
ramyav7796
by New Contributor II
  • 474 Views
  • 2 replies
  • 1 kudos

Databricks Lakehouse Monitoring

Hi,I am trying to implement lakehouse monitoring using Inference profile for my inference data that I have, I see that when I create the monitor, two tables get generated profile and drift, I wanted to understand how are these two tables generating a...

  • 474 Views
  • 2 replies
  • 1 kudos
Latest Reply
BigRoux
Databricks Employee
  • 1 kudos

When you create a Databricks Lakehouse Monitoring monitor with an Inference profile, the system automatically generates two metric tables: a profile metrics table and a drift metrics table. Here's how this process works: Background Processing When yo...

  • 1 kudos
1 More Replies
ShivangiB
by New Contributor III
  • 595 Views
  • 2 replies
  • 0 kudos

Liquid Clustering Key Change Question

If i already have a cluster key1 for existing table, i want to change cluster key to key2 using ALTER TABLE table CLUSTER BY (key2), then run OPTIMIZE table, based on databrick document , existing files will not be rewritten (verified by my test as w...

  • 595 Views
  • 2 replies
  • 0 kudos
Latest Reply
LRALVA
Honored Contributor
  • 0 kudos

@ShivangiB You're correct in your understanding. When you change a clustering key using ALTER TABLE followed by OPTIMIZE, it doesn't automatically recluster existing data. Let me explain why this happens and what options you have.In Delta Lake (which...

  • 0 kudos
1 More Replies
HarryRichard08
by New Contributor II
  • 440 Views
  • 1 replies
  • 0 kudos

Unable to Access S3 from Serverless but Works on Cluster

Hi everyone,I am trying to access data from S3 using an access key and secret. When I run the code through Databricks clusters, it works fine. However, when I try to do the same from a serverless cluster , I am unable to access the data.I have alread...

  • 440 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika
Databricks Employee
  • 0 kudos

Hello @HarryRichard08! It looks like this post duplicates the one you recently posted. A response has already been provided to the Original post. I recommend continuing the discussion in that thread to keep the conversation focused and organized.

  • 0 kudos
December
by New Contributor II
  • 804 Views
  • 0 replies
  • 0 kudos

NiFi on EKS Fails to Connect to Databricks via JDBC – "Connection reset" Error

I'm using Apache Nifi (running on AWS EKS) to connect to Databricks (with compute on EC2) via JDBC. My JDBC URL is as follows: jdbc:databricks://server_hostname:443/default;transportMode=http;ssl=1;httpPath=my_httppath;AuthMech=3;UID=token;PWD=my_tok...

December_1-1741684901117.png
Get Started Discussions
Connection
JDBC
  • 804 Views
  • 0 replies
  • 0 kudos
Fasih_Ahmed
by New Contributor III
  • 2973 Views
  • 4 replies
  • 0 kudos

Resolved! Exam suspended due to sudden power cut

Hi @Cert-Team   I hope this message finds you well. I am writing to request a review of my recently suspended exam. I believe that my situation warrants reconsideration, and I would like to provide some context for your understanding.I applied for Da...

  • 2973 Views
  • 4 replies
  • 0 kudos
Latest Reply
Cert-Bricks
Databricks Employee
  • 0 kudos

This has been resolved. 

  • 0 kudos
3 More Replies
Lackshu
by New Contributor II
  • 1720 Views
  • 2 replies
  • 0 kudos

Workspace Assignment Issue via REST API

I’m relying on workspace assignment via REST API to have the account user created in the workspace. This is like the workspace assignment screen at account level or adding existing user screen at workspace level. The reference URL is below.Workspace ...

  • 1720 Views
  • 2 replies
  • 0 kudos
Latest Reply
Lackshu
New Contributor II
  • 0 kudos

It turns out, the problem is the documentation. It says that the permission parameter (that's supplied in) is an array of strings. It really just expects a string, either UNKNOWN, USER, or ADMIN. It would be great if the team could fix the documentat...

  • 0 kudos
1 More Replies
gauravmahajan
by New Contributor II
  • 433 Views
  • 3 replies
  • 0 kudos

Require Information on SQL Analytics DBU Cluster

Hello TeamWe are seeking cost information as we have noticed fluctuations in the daily costs for the "SQL Analytics DBU." We would like to understand the reasons behind the daily cost differences, even though the workload remains consistent.trying to...

  • 433 Views
  • 3 replies
  • 0 kudos
Latest Reply
Nivethan_Venkat
Contributor II
  • 0 kudos

Hi @gauravmahajan,Most of the cost / DBU used can be retrieved from System tables across your different workspaces in a databricks account. Details related to job compute types and it's associated cost can be fetched from the queries mentioned in the...

  • 0 kudos
2 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels