- 5152 Views
- 2 replies
- 3 kudos
Resolved! Limitations of committing ipynb notebook output with Repos
During my experimentation with the latest feature that allows including notebook output in a commit, I ran into a specific issue. While attempting to commit my recent changes, I encountered an error message stating "Error fetching Git status." Intere...
- 5152 Views
- 2 replies
- 3 kudos
- 3 kudos
I've found that the restriction I've encountered isn't related to the file size within Repos, but rather the maximum file size that can be shown in the Azure Databricks UI. You can find this limitation documented at https://learn.microsoft.com/en-us/...
- 3 kudos
- 8282 Views
- 1 replies
- 1 kudos
Structured Streaming of S3 source
I am trying to setup s3 as a structured streaming source. The bucket receives ~17K files/day and the original load to the bucket was ~54K files. The bucket was first loaded 3 months ago and we haven't started reading from it since. So let's say there...
- 8282 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks,We were able to make things work by increasing the driver instance size so it has more memory for the initial load. After initial load we scaled the instance down for subsequent runs. We're still testing, if we aren't able to make it work we'l...
- 1 kudos
- 1412 Views
- 0 replies
- 0 kudos
"command complete" but not executed
I have a notebook that calls other notebooks with `dbutils.notebook.run` and execute them as a 'Notebook job'. But sometimes when a notebook is taking a long time and the cluster is just waiting for, for instance, an api response, the subsequent comm...
- 1412 Views
- 0 replies
- 0 kudos
- 31250 Views
- 8 replies
- 6 kudos
Get up to speed on Generative AI with this free on-demand training
Build foundational knowledge of generative AI, including large language models (LLMs), with 4 short videos. Here is how it works: Watch 4 short tutorial videosPass the knowledge testEarn a badge for Generative AI Fundamentals you can share on your ...
- 31250 Views
- 8 replies
- 6 kudos
- 7282 Views
- 3 replies
- 0 kudos
Databricks Certified Data Engineer Associate exam
My Databricks Certified Data Engineer Associate exam got timed out as I was uninstalling the Antivirus, it took a while to set up my device. By that time it got timed out to start the exam. So please reschedule my exam.webassessor id: anjaliisanvaga...
- 7282 Views
- 3 replies
- 0 kudos
- 0 kudos
Dear Support Team, I apologize for the inconvenience caused. Unfortunately, due to unforeseen circumstances with my antivirus uninstallation, I was unable to start the exam on time. I kindly request you to reschedule my Databricks Certified Data Engi...
- 0 kudos
- 7273 Views
- 3 replies
- 0 kudos
Data Engineer Associate V3 exam got suspended
Hi team,My Databricks Certified Data Engineer Associate exam got suspended within 40 minutes. I had also shown my exam room to the proctor. My exam got suspended due to eye movement. I was not moving my eyes away from laptop screen. Some questions ar...
- 7273 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @vivek2612 the support team did receive your case and is working on it now. Good luck on your exam!
- 0 kudos
- 2254 Views
- 0 replies
- 0 kudos
Cluster can't connect to redshift
I'm trying to connect to redshift with redshift-connector module, but it get 'Connection refused' error, redshift cluster and ec2 cluster are in the same VPC, security group of redshift cluster is open with inbound from VPC's IP range, i can connect ...
- 2254 Views
- 0 replies
- 0 kudos
- 8450 Views
- 0 replies
- 0 kudos
Left join with merging two streaming tables returns inner join instead
Hi all, Im trying to merge two streaming tables together with a left (outer) join, but it seems to somehow return all values from the left table that have matching values from the right table, instead of also appending the NULL values if there is no ...
- 8450 Views
- 0 replies
- 0 kudos
- 1806 Views
- 0 replies
- 0 kudos
Mounting adls gen2 from databricks RBAC issue
I am trying to mount my Gen 2 storage account in databricks. I have added permission of Storage blob data contributor to the storage account. But I am getting below error:Invalid permissions on the specified KeyVault https://kvmigrationnew.vault.azu...
- 1806 Views
- 0 replies
- 0 kudos
- 2911 Views
- 1 replies
- 0 kudos
Ydata-profiling on Spark cluster -working for you?
I am trying to profile my dataset using ydata-profiling.I constantly run into errors, even with simple datasets on my spark cluster.Profiling in Spark cluster erroring out · Issue #1350 · ydataai/ydata-profiling (github.com)Has anyone got ydata-profi...
- 2911 Views
- 1 replies
- 0 kudos
- 1700 Views
- 2 replies
- 0 kudos
Mosaic library not supported in db13 and unity catalog
I want to use gepatial functions which uses mosaic library. How can I use it without using mosaic library and import functions available ?Example st_aswkt
- 1700 Views
- 2 replies
- 0 kudos
- 6068 Views
- 1 replies
- 2 kudos
Using Azure Event Grid for structured streaming
Can anyone point me to any Databricks documentation (or other resources) for configuring structured streaming to use Azure Event Grid for a source/sink? I found examples for Kafka and EventHubs but Azure Event Grid is different than Azure Event Hubs....
- 6068 Views
- 1 replies
- 2 kudos
- 2 kudos
I must be missing something. I don't see how the examples in the referenced document can be applied to Azure Event Grid.Is there another example that shows how to subscribe to an Event Grid topic?
- 2 kudos
- 1024 Views
- 1 replies
- 0 kudos
sql end point and jdbc driver
When we try and connect to a sql warehouse endpoint with the databricks jdbc driver our query is failing if we use first_value(). We've rewritten the query to use limit 1, but we would like to understand if this is a gap in the simba/databricks driv...
- 1024 Views
- 1 replies
- 0 kudos
- 0 kudos
A sample error message when using first_value() is:An error occurred while calling o132.csv. [Databricks][JDBC](10140) Error converting value to BigDecimal.
- 0 kudos
- 1172 Views
- 1 replies
- 0 kudos
Databricks certified Data engineer Associate exam got suspended_need immediate help(10/09/2023)
Hi team,I've scheduled my exam on 10th September,2023 at 15:15hrs Asia Calcutta timing but my exam got suspended stating no proper environment by the proctor.Please check this issue as I didn't attempt one single question too, this is not fair as I l...
- 1172 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @sirishavemula20 We have addressed the case. If the issue is not resolved, please reply back to the case notes.
- 0 kudos
- 546 Views
- 0 replies
- 0 kudos
Hive Metastore permission on DBX 10.4
I've been working on creating a schema in the Hive Metastore using the following command:spark.sql(f'CREATE DATABASE IF NOT EXISTS {database}')The schema or database is successfully created, but I encountered an issue where it's only accessible for m...
- 546 Views
- 0 replies
- 0 kudos
