cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Ender
by New Contributor II
  • 371 Views
  • 3 replies
  • 1 kudos

Resolved! Accessing ADLS Gen2 related Hadoop configuration in notebook

I have a cluster in which I have the required configuration to access an ADLS Gen2, and it works without any problems.I want to access this storage using the Hadoop filesystem APIs. To achieve this, I am trying to get the Hadoop configuration from th...

Ender_1-1717335940727.png Ender_3-1717336689720.png
  • 371 Views
  • 3 replies
  • 1 kudos
Latest Reply
Ender
New Contributor II
  • 1 kudos

By the way how do you achieve inline code highlighting in the editor I tried `` but it didn't work.

  • 1 kudos
2 More Replies
Shawn_Eary
by New Contributor III
  • 168 Views
  • 1 replies
  • 2 kudos

Resolved! DAIS24 Attendee Badge - Oops

I wasn't at DAIS24, but I received one of these emails and it appears to have come from Databricks. I think I was given the badge in error. Can we have it removed?Just Wondering,Shawn

Shawn_Eary_1-1717680651702.png
  • 168 Views
  • 1 replies
  • 2 kudos
Latest Reply
Sujitha
Community Manager
  • 2 kudos

Hi @Shawn_Eary Thank you for bringing this to our attention. We were conducting a test on badges, and some of our community members may have received this email by accident. We have revoked the changes now, please ignore the message.

  • 2 kudos
ravitejasutrave
by New Contributor
  • 122 Views
  • 1 replies
  • 0 kudos

Databricks + confluent schema registry - Schema not found error

I am running a Kafka producer code on Databricks 12.2. I am testing AVRO serialization of message with help of confluent schema registry. I configured 'to_avro' function to read the schema from schema registry, but I am getting the below error> org.a...

  • 122 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @ravitejasutrave,  Ensure that the schema is compatible with the data you’re trying to serialize.Double-check your configuration for connecting to the schema registry. Make sure that the schemaRegistryAddress points to the correct URL where your s...

  • 0 kudos
PrathviS
by New Contributor
  • 179 Views
  • 1 replies
  • 0 kudos

Training link deprecated: How to ingest data for Databricks SQL

I am currently doing a course in Databricks academy: How to ingest data for Databricks SQL.To create a table in the external location I am provided with the link that is not working anymore. Below is the link:wasbs://courseware@dbacademy.blob.core.wi...

  • 179 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @PrathviS, Thank you for sharing your concern with us!   To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours).

  • 0 kudos
traillog
by New Contributor
  • 126 Views
  • 1 replies
  • 0 kudos

Unable to unzip files recursively and copy into a different folder

I am currently trying to unzip files recursively from one folder(source folder) and copy all the unzipped files into the destination folder using databricks(pyspark). The destination path is still empty even after running this code. I tried looking f...

  • 126 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @traillog,  To recursively unzip files from a source folder, you can use the os.walk() function to traverse through all subdirectories and files.Your current implementation only processes the top-level directory. To handle recursion, you need to i...

  • 0 kudos
vabadzhiev
by New Contributor II
  • 361 Views
  • 2 replies
  • 0 kudos

Tableau Prep Save Output to Databricks

Has anyone run into use cases where your data scientist/data engineer end users build Tableau Prep Flows and steps in Tableau Prep Flow require saving output back into Databricks? There appears to be no native support for this in Tableau Prep if the ...

  • 361 Views
  • 2 replies
  • 0 kudos
Latest Reply
vabadzhiev
New Contributor II
  • 0 kudos

These are awesome suggestions. To expand on our setup, we also have Informatica Cloud - IICS (CMI, CDI, etc.) connected to the entire setup generally used for bringing data from a source (PaaS, SaaS, On-prem SQL, Flat Files or streaming devices) to D...

  • 0 kudos
1 More Replies
kazinahian
by New Contributor III
  • 2456 Views
  • 3 replies
  • 1 kudos

Seeking Tips: Ways to Master Databricks on Azure?

Hello everyone. I'm currently learning Databricks on Azure through a Udemy course. Recently, I was surprised by a charge of $86 from Azure, which has made me cautious about continuing in this manner. Is there a more cost-effective approach to learn D...

  • 2456 Views
  • 3 replies
  • 1 kudos
Latest Reply
LauraMurphy
New Contributor II
  • 1 kudos

Thank you so much for the information.

  • 1 kudos
2 More Replies
Kushal12345
by New Contributor III
  • 6199 Views
  • 15 replies
  • 1 kudos

DataBricks Certification Exam Got Suspended. Require support for the same.

Hello Team, I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam, saying I have exceeded ...

  • 6199 Views
  • 15 replies
  • 1 kudos
Latest Reply
Aravind7
New Contributor II
  • 1 kudos

@Cert-Team Same issue happened with me 72 hours ago, but still no response. I am hoping for a response ASAP. Please assist regarding this.Email id : @aravindmarrapu7901@gmail.comCertification: Databricks Certified Data Engineer AssociateTicket id : #...

  • 1 kudos
14 More Replies
samarth10
by New Contributor II
  • 104 Views
  • 1 replies
  • 0 kudos

Assigning a group as USER to service principal

How can we assigning a group as USER to service principal using databricks-sdk, this is not supported?

  • 104 Views
  • 1 replies
  • 0 kudos
Latest Reply
samarth10
New Contributor II
  • 0 kudos

I found this API, https://docs.databricks.com/api/account/accountaccesscontrol/updaterulesetbut its PUT and GET method both requires a parameter "etag", how can someone know this "etag"?

  • 0 kudos
Avvar2022
by Contributor
  • 1659 Views
  • 6 replies
  • 1 kudos

Unity catalog enabled workspace -Is there any way to disable workflow/job creation for certain users

Currently in unity catalog enabled workspace users with "Workspace access" can create workflows/jobs, there is no access control available to restrict users from creating jobs/workflows.Use case: In production there is no need for users, data enginee...

  • 1659 Views
  • 6 replies
  • 1 kudos
Latest Reply
Avvar2022
Contributor
  • 1 kudos

not being able to restrict creation of workflows/jobs, alerts, dashboards make platform admin job difficult to keep system clean and control cost. there is no need for data engineers to create a workflow in production. there is no need for all users ...

  • 1 kudos
5 More Replies
Henrik
by New Contributor III
  • 221 Views
  • 3 replies
  • 0 kudos

List granted access for a group or a user

Is there any way where I can see what access a group or a user have been given to objects (Tables, views, catalogs etc.)?I noticed that we have the following information_schema tables:catalog_privilegesroutine_privilegestable_privilegesschema_privile...

  • 221 Views
  • 3 replies
  • 0 kudos
Latest Reply
anardinelli
New Contributor III
  • 0 kudos

Hi @Henrik  To grant a user the privilege to query system tables, a metastore admin or another privileged user must grant USE and SELECT permissions on the system schemas.   GRANT USAGE ON CATALOG system TO <user_name>; GRANT USAGE ON SCHEMA informa...

  • 0 kudos
2 More Replies
Kasen
by New Contributor II
  • 143 Views
  • 1 replies
  • 0 kudos

Materialized view in DLT pipeline

When setting up DLT pipeline, there are 3 types of product edition, Core, Pro and Advanced. When I compare DLT Classic Core and DLT Classic Pro, the difference is that DLT Classic Pro can handle CDC. Does it means if I'm using DLT Classic Core, I hav...

Community Discussions
DLT pipeline
Materialized view
  • 143 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kasen
New Contributor II
  • 0 kudos

Also, If I'm using DLT Classic Core for DLT pipeline, the materialized view will be doing a full refresh or just update on the rows that have changes? 

  • 0 kudos
SHASHANK2
by New Contributor II
  • 356 Views
  • 3 replies
  • 1 kudos

Resolved! Test has been suspended

Hello guys,I was taking test for data bricks data analyst associate and irrespective of any solid reason they have suspended my test. saying i am not giving eyecontact to camera. how can i read questions and options while eyecontacting with camera. I...

  • 356 Views
  • 3 replies
  • 1 kudos
Latest Reply
SHASHANK2
New Contributor II
  • 1 kudos

@Cert-TeamOPS Thanks for you timely support, my issue has been cleared. my test has been rescheduled to the time I have asked. I am very grateful for your responses.Thanks and Regards@SHASHANK2 

  • 1 kudos
2 More Replies
young_sheldon
by New Contributor
  • 182 Views
  • 1 replies
  • 0 kudos

How to optimize a View ?

I have create a view from my underlying delta live table. Generally delta live table get auto optimize with vaccum in place. If thats so why do my view take 1 hour to be queried ? Any other way to optimize it ?

  • 182 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @young_sheldon,  Ensure that your table is properly partitioned and clustered. Partitioning by relevant columns (such as device_id) can significantly speed up queries and reduce data shuffling during reads.Clustering the data based on frequently a...

  • 0 kudos
SHASHANK2
by New Contributor II
  • 470 Views
  • 4 replies
  • 2 kudos

Resolved! About resuming the test - [ ref:_00D61JGc4._500Vp6jMk5:ref ]

Hello @Cert-Team I am very glad to hear from you,in response of the ticket (00481969), I have received mail for rescheduling of my test to resume. I am very thankful for your quick response in this matter.I have replied in mail about the date and tim...

  • 470 Views
  • 4 replies
  • 2 kudos
Latest Reply
SHASHANK2
New Contributor II
  • 2 kudos

@Cert-Team, My issue has been clear. my test has been rescheduled to the time I have mentioned.I am very grateful for your help.Thanks and Regards@SHASHANK2 

  • 2 kudos
3 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!