cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Community Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
SainnathReddy_M
by • New Contributor II
  • 131 Views
  • 2 replies
  • 0 kudos

Monitoring Databricks

Good Day All, Did any one done Databricks Monitoring integration with  any other 3rd party applications like Grafana or ELK to get infrastructure monitoring like cpu_utilization ,memory ,job monitor and I can able to write spark code to get cpu_utili...

Community Discussions
Reply
  • 131 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @SainnathReddy_M , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your...

  • 0 kudos
1 More Replies
HemanthG
by • New Contributor
  • 125 Views
  • 2 replies
  • 0 kudos

Unable to install databricks free trial on AWS

I was trying to install databricks on AWS using these instructions "https://docs.databricks.com/en/admin/workspace/quick-start.html#use-aws-quickstart-to-create-a-databricks-workspace". But when CloudFormation Template stack executed "createCredentia...

Community Discussions
Reply
  • 125 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @HemanthG , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedba...

  • 0 kudos
1 More Replies
v_sravan_sai
by • New Contributor
  • 105 Views
  • 2 replies
  • 0 kudos

UnsupportedOperationException: com.databricks.backend.daemon.data.client.DBFSV1.resolvePathOnPhysica

dbutils.fs.mv('dbfs:/FileStore/tables/Employee-2.csv','dbfs:/FileStore/backup/Employee-5.csv',recurse=True)--is giving errorUnsupportedOperationException: com.databricks.backend.daemon.data.client.DBFSV1.resolvePathOnPhysicalStorage(path: Path)File <...

Community Discussions
Reply
  • 105 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @v_sravan_sai , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your fe...

  • 0 kudos
1 More Replies
databricksdummy
by • New Contributor
  • 173 Views
  • 2 replies
  • 0 kudos

INVALID_PARAMETER_VALUE: There are more than 1001 files. Please specify [LIMIT] to limit the num....

I am running Azure Databricks DBX Runtime 13.3 and I am seeing the following error in my workspace:Last execution failed# doesn't work in DBX Runtime 13.3 and above# limitation is it can't read more than 1001 filesdbutils.fs.ls("[path]")AnalysisExcep...

Community Discussions
Reply
  • 173 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @databricksdummy , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your...

  • 0 kudos
1 More Replies
Dorothy80Galvin
by • New Contributor II
  • 418 Views
  • 2 replies
  • 1 kudos

How can I Resolve QB Desktop Update Error 15225?

I'm encountering QB Desktop update error 15225. What could be causing this issue, and how can I resolve it? It's disrupting my workflow, and I need a quick fix.

Community Discussions
Reply
  • 418 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
jameshardy602
New Contributor II
  • 1 kudos

hi @Dorothy80Galvin To resolve Desktop Update Error 15225, follow these steps. First, verify your Internet Explorer settings by ensuring it is set as the default browser and that SSL settings are enabled. Next, add trusted sites by navigating to Inte...

  • 1 kudos
1 More Replies
paras11
by • New Contributor II
  • 235 Views
  • 4 replies
  • 1 kudos

Databricks data engineer associate exam Suspended

Hi Team,I recently had a disappointing experience while attempting my first Data bricks certification exam. During the exam, I was abruptly directed to Proctor Support. The proctor asked me to show my desk and the room I was in. I complied by showing...

Community Discussions
@Cert-Bricks@Cert-Team@Cert-TeamOPS @Kaniz_Fatma
Reply
  • 235 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
paras11
New Contributor II
  • 1 kudos

 @Kaniz_Fatma @Cert-Team @Cert-Bricks Requesting you to please look into this and update me since it has not been resolved yet and I am not able to reschedule my exam. 

  • 1 kudos
3 More Replies
TinaN
by • New Contributor II
  • 274 Views
  • 3 replies
  • 3 kudos

Resolved! Extracting 'time' from a 'timestamp' datatype in Databricks

We are loading a data source to Databricks that contains columns with 'Time' datatype.  Databricks converts this to 'Timestamp', so I am researching for a way to extract time only. This is what I came up with, but the result isn't quite right.  Is th...

Community Discussions
Reply
  • 274 Views
  • 3 replies
  • 3 kudos
View Replies
Latest Reply
Slash
New Contributor II
  • 3 kudos

Hi @TinaN ,I check it in the evening, but try below: SELECT date_format(timestamp_column, 'HH:mm:ss') AS time_partFROM your_table

  • 3 kudos
2 More Replies
Prasad_Koneru
by • New Contributor III
  • 264 Views
  • 4 replies
  • 0 kudos

Deployment of tables and views in unity catalog and Repo structuring for catalogs objects.

we want to create the CI/CD Pipeline for deploying Unity catalog objects inorder to enhance the deployment ability.but How to maintain a repository of Tables/views/ or any other objects created in the catalogs and schema.Is this possible to do just l...

Community Discussions
Reply
  • 264 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @Prasad_Koneru , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your f...

  • 0 kudos
3 More Replies
icyapple
by • New Contributor
  • 198 Views
  • 3 replies
  • 0 kudos

hive_metastore schema access control

We are trying to control access to schemas under hive_metastore, only allowing certain users to access the tables under a schema (via SQL, Pyspark, and Python...), we have follow steps in a testing schema:1. Enable workspace table access control2. Ru...

Community Discussions
Reply
  • 198 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @icyapple , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedba...

  • 0 kudos
2 More Replies
SHASHANK2
by • New Contributor III
  • 236 Views
  • 3 replies
  • 1 kudos

cluster termination

Hello All,when I am creating all purpose cluster I am getting an idle time of 3 days, my cluster is terminating after 3days, I want to make my cluster terminate in 60 min of idle time, i want to do it globally so that in future any cluster created by...

Community Discussions
Reply
  • 236 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 1 kudos

Hi @SHASHANK2 , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...

  • 1 kudos
2 More Replies
alluarjun
by • New Contributor
  • 134 Views
  • 2 replies
  • 0 kudos

databricks asset bundle error-terraform.exe": file does not exist

Hi,I am getting below error while I am deploying databricks bundle using azure devops release  2024-07-07T03:55:51.1199594Z Error: terraform init: exec: "xxxx\\.databricks\\bundle\\dev\\terraform\\xxxx\\.databricks\\bundle\\dev\\bin\\terraform.exe": ...

Community Discussions
Reply
  • 134 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Rishabh_Tiwari
Community Manager
  • 0 kudos

Hi @alluarjun , Thank you for reaching out to our community! We're here to help you.  To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedb...

  • 0 kudos
1 More Replies
Yashodip
by • New Contributor II
  • 170 Views
  • 2 replies
  • 0 kudos

My Databricks Professional Data Engineer exam has suspended , Need help Urgently (17/07/2024)

Hello Team, I encountered Pathetic experience while attempting my Professional Data Engineer DataBricks certification. Abruptly, Proctor asked me to show my desk, after 30 mins of exam showing he/she asked multiple times.. wasted my time and then sus...

Community Discussions
Reply
  • 170 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Yashodip, I'm sorry to hear your exam was suspended. Thank you for filing a ticket with our support team. Please allow the support team 24-48 hours to resolve. In the meantime, you can review the following documentation: Room requirements Behavio...

  • 0 kudos
1 More Replies
elsirya
by • New Contributor II
  • 144 Views
  • 2 replies
  • 2 kudos

Resolved! unit testing

Currently I am creating unit tests for our ETL scripts although the test is not able to recognize sc (SparkContext).Is there a way to mock SparkContext for a unit test? Code being tested: df = spark.read.json(sc.parallelize([data])) Error message rec...

Community Discussions
Reply
  • 144 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
elsirya
New Contributor II
  • 2 kudos

Was able to get this to work.What I had to do was instantiate the "sc" variable in the PySpark notebook.PySpark code:"sc = spark.SparkContext"Then in the PyTest script we add a "@patch()" statement with the "sc" variable and create a "mock_sc" variab...

  • 2 kudos
1 More Replies
aditi06
by • New Contributor II
  • 81 Views
  • 2 replies
  • 1 kudos

Issue while launching the data engineer exam

I started my exam but it said due to a technical issue it had been suspended though I checked all prerequisites and system checks.I have already raised the ticket please resolve this issue as early as possible.Ticket no.:  #00504757#reschedule #issue...

Community Discussions
Reply
  • 81 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
aditi06
New Contributor II
  • 1 kudos

@Kaniz_Fatma , Please resolve this issue as early as possible. I have already raised the ticket

  • 1 kudos
1 More Replies
Simon_T
by • New Contributor III
  • 439 Views
  • 3 replies
  • 1 kudos

Resolved! Databricks Bundle Error

I am running this command: databricks bundle deploy --profile DAVE2_Dev --debug And I am getting this error: 10:13:28 DEBUG open dir C:\Users\teffs.THEAA\OneDrive - AA Ltd\Databricks\my_project\dist: open C:\Users\teffs.THEAA\OneDrive - AA Ltd\Databr...

Community Discussions
Reply
  • 439 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Simon_T
New Contributor III
  • 1 kudos

So I found a link to a page that said that the databricks bundle command is expecting python3.exe instead of python.exe. So I took a copy of python.exe and renamed it to python3.exe and that seems to work. Thanks for investigating though.

  • 1 kudos
2 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels
Top Kudoed Authors
View all