cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
eimis_pacheco
by • Contributor
  • 3926 Views
  • 2 replies
  • 2 kudos

Resolved! Is it not needed to preserve the data in its original format anymore with the usage of medallion?

Hi Community I have a doubt. The bronze layer always causes confusion for me. Someone mentioned, "File Format: Store data in Delta Lake format to leverage its performance, ACID transactions, and schema evolution capabilities" for bronze layers.Then, ...

Community Platform Discussions
Reply
  • 3926 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
<div style= This widget could not be displayed.
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 2 kudos

This widget could not be displayed.
Hi Community I have a doubt. The bronze layer always causes confusion for me. Someone mentioned, "File Format: Store data in Delta Lake format to leverage its performance, ACID transactions, and schema evolution capabilities" for bronze layers.Then, ...

This widget could not be displayed.
  • 2 kudos
This widget could not be displayed.
1 More Replies
vivek2612
by • New Contributor II
  • 8533 Views
  • 5 replies
  • 0 kudos

Exam got suspended Databricks Certified Data Engineer Associate exam

Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 20 minutes.  My exam got suspended due to eye movement without any warning. I was not moving my eyes away from laptop screen. Some questions are so big in the exam so I...

Community Platform Discussions
Reply
  • 8533 Views
  • 5 replies
  • 0 kudos
View Replies
Latest Reply
Cert-Team
Databricks Employee
  • 0 kudos

@rajib_bahar_ptg funny, not funny, right?! I just posted that tip today in this post: https://community.databricks.com/t5/certifications/minimize-the-chances-of-your-exam-getting-suspended-tip-3/td-p/45712

  • 0 kudos
4 More Replies
Mbinyala
by • New Contributor II
  • 5121 Views
  • 6 replies
  • 3 kudos

Cluster policy not showing while creating delta live table pipeline

Hi all!!!I have created a cluster policy but when i want to use that while creating dlt pipeline, It is showing none. I have checked I have all the necessary permissions to create cluster policies. Still, in dlt ui it is showing none.

Community Platform Discussions
Reply
  • 5121 Views
  • 6 replies
  • 3 kudos
View Replies
Latest Reply
Rishitha
New Contributor III
  • 3 kudos

@btafur  Can we also set the auto_terminate minutes with the policy?  (for the dlt cluster type)

  • 3 kudos
5 More Replies
RithwikMR
by • New Contributor
  • 1048 Views
  • 1 replies
  • 0 kudos

How to automate creating notebooks when i have multiple .html or .py files

Hi all,I have 50+ .html and .py files, for which I have to create separate notebooks for each and every single one of them. For this, manually creating a notebook using the UI and importing the .html/.py file is a bit tedious and time consuming. Is t...

Community Platform Discussions
Reply
  • 1048 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
btafur
Databricks Employee
  • 0 kudos

Depending on your use case and requirements, one alternative would be to create a script that loops through your files and uploads them using the API. You can find more information about the API here: https://docs.databricks.com/api/workspace/workspa...

  • 0 kudos
adrianna2942842
by • New Contributor III
  • 4826 Views
  • 2 replies
  • 3 kudos

Resolved! Limitations of committing ipynb notebook output with Repos

During my experimentation with the latest feature that allows including notebook output in a commit, I ran into a specific issue. While attempting to commit my recent changes, I encountered an error message stating "Error fetching Git status." Intere...

Community Platform Discussions
Reply
  • 4826 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
adrianna2942842
New Contributor III
  • 3 kudos

I've found that the restriction I've encountered isn't related to the file size within Repos, but rather the maximum file size that can be shown in the Azure Databricks UI. You can find this limitation documented at https://learn.microsoft.com/en-us/...

  • 3 kudos
1 More Replies
dvmentalmadess
by • Valued Contributor
  • 8085 Views
  • 1 replies
  • 1 kudos

Structured Streaming of S3 source

I am trying to setup s3 as a structured streaming source. The bucket receives ~17K files/day and the original load to the bucket was ~54K files. The bucket was first loaded 3 months ago and we haven't started reading from it since. So let's say there...

Community Platform Discussions
Reply
  • 8085 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
dvmentalmadess
Valued Contributor
  • 1 kudos

Thanks,We were able to make things work by increasing the driver instance size so it has more memory for the initial load. After initial load we scaled the instance down for subsequent runs. We're still testing, if we aren't able to make it work we'l...

  • 1 kudos
jl1
by • New Contributor
  • 1336 Views
  • 0 replies
  • 0 kudos

"command complete" but not executed

I have a notebook that calls other notebooks with `dbutils.notebook.run` and execute them as a 'Notebook job'. But sometimes when a notebook is taking a long time and the cluster is just waiting for, for instance, an api response, the subsequent comm...

Community Platform Discussions
Reply
  • 1336 Views
  • 0 replies
  • 0 kudos
Sujitha
by • Databricks Employee
  • 30770 Views
  • 8 replies
  • 6 kudos

Get up to speed on Generative AI with this free on-demand training

Build foundational knowledge of generative AI, including large language models (LLMs), with 4 short videos.   Here is how it works: Watch 4 short tutorial videosPass the knowledge testEarn a badge for Generative AI Fundamentals you can share on your ...

Screenshot 2023-07-19 at 11.29.11 AM.png
Community Platform Discussions
Reply
  • 30770 Views
  • 8 replies
  • 6 kudos
View Replies
Latest Reply
rahmat_kun
New Contributor II
  • 6 kudos

Videos not running

  • 6 kudos
7 More Replies
Anjali123
by • New Contributor
  • 7154 Views
  • 3 replies
  • 0 kudos

Databricks Certified Data Engineer Associate exam

My Databricks Certified Data Engineer Associate exam got timed out as I was uninstalling the Antivirus, it took a while to set up my device. By that time it got timed out to start the exam. So please reschedule my exam.webassessor id:  anjaliisanvaga...

Community Platform Discussions
Reply
  • 7154 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
victdarcel09
New Contributor II
  • 0 kudos

Dear Support Team, I apologize for the inconvenience caused. Unfortunately, due to unforeseen circumstances with my antivirus uninstallation, I was unable to start the exam on time. I kindly request you to reschedule my Databricks Certified Data Engi...

  • 0 kudos
2 More Replies
vivek2612
by • New Contributor II
  • 7031 Views
  • 3 replies
  • 0 kudos

Data Engineer Associate V3 exam got suspended

Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 40 minutes. I had also shown my exam room to the proctor. My exam got suspended due to eye movement. I was not moving my eyes away from laptop screen. Some questions ar...

Community Platform Discussions
Reply
  • 7031 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Cert-Team
Databricks Employee
  • 0 kudos

Hi @vivek2612 the support team did receive your case and is working on it now. Good luck on your exam!

  • 0 kudos
2 More Replies
Danh_Hoa
by • New Contributor
  • 2130 Views
  • 0 replies
  • 0 kudos

Cluster can't connect to redshift

I'm trying to connect to redshift with redshift-connector module, but it get 'Connection refused' error, redshift cluster and ec2 cluster are in the same VPC, security group of redshift cluster is open with inbound from VPC's IP range, i can connect ...

Community Platform Discussions
cluster
redshift-connector
Reply
  • 2130 Views
  • 0 replies
  • 0 kudos
Brammer88
by • New Contributor III
  • 8212 Views
  • 0 replies
  • 0 kudos

Left join with merging two streaming tables returns inner join instead

Hi all, Im trying to merge two streaming tables together with a left (outer) join, but it seems to somehow return all values from the left table that have matching values from the right table, instead of also appending the NULL values if there is no ...

Community Platform Discussions
Reply
  • 8212 Views
  • 0 replies
  • 0 kudos
keer1392
by • New Contributor
  • 1703 Views
  • 0 replies
  • 0 kudos

Mounting adls gen2 from databricks RBAC issue

 I am trying to mount my Gen 2 storage account in databricks. I have added permission of Storage blob data contributor to the storage account. But I am getting below error:Invalid permissions on the specified KeyVault https://kvmigrationnew.vault.azu...

Community Platform Discussions
Reply
  • 1703 Views
  • 0 replies
  • 0 kudos
manojvas
by • New Contributor II
  • 2672 Views
  • 1 replies
  • 0 kudos

Ydata-profiling on Spark cluster -working for you?

I am trying to profile my dataset using ydata-profiling.I constantly run into errors, even with simple datasets on my spark cluster.Profiling in Spark cluster erroring out · Issue #1350 · ydataai/ydata-profiling (github.com)Has anyone got ydata-profi...

Community Platform Discussions
Reply
  • 2672 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
<div style= This widget could not be displayed.
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
I am trying to profile my dataset using ydata-profiling.I constantly run into errors, even with simple datasets on my spark cluster.Profiling in Spark cluster erroring out · Issue #1350 · ydataai/ydata-profiling (github.com)Has anyone got ydata-profi...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels
Top Kudoed Authors
View all