Hi, Databricks team,I created the share and provided access to the recipient, and the recipient is consuming or accessing the share. My question is who is going to bear the cost and how can we track the computation charges for it?As per the attached ...
Hi @Kaniz Fatma​ , Thanks for your response on Delta sharing information, Could you please provide details about who is going to bear the cost[either provider/Consumer] and how can we track the computation charges for it?
Hi Team,I am asking a little off topic question here. I have been attending paid session for Data AI summit in last 2-3 years. Is it possible to get videos for sessions which we have not attended due to session conflicts at the same time. I mean to s...
I have configured a Databricks job to send email alters to me whenever my job fails. However, I would very much like to alter the text in the Alert Email to something a little more bespoke. Is there any way to alter the text in email or even just t...
Hey there @Peter Mayers​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from yo...
Hi,I have datalake gen2 with vnet and private endpoint. I do have databricks workspace in same vnet. I am trying to access the datalake from databricks but I keep getting error when I allow access only for selected network in datalake. I get error w...
Currently I am having some issues with the writing of the parquet file in the Storage Container. I do have the codes running but whenever the dataframe writer puts the parquet to the blob storage instead of the parquet file type, it is created as a f...
Hello @Karl Saycon​ Can you try setting this config to prevent additional parquet summary and metadata files from being written? The result from dataframe write to storage should be a single file.https://community.databricks.com/s/question/0D53f00001...
We have created our own artifactory and we use this to install python dependencies or libraries.We would like to know how we can make use of our own artifactory to install dependencies or libraries on Databricks clusters..
For private repos, you can find some good examples herehttps://kb.databricks.com/clusters/install-private-pypi-repo.htmlhttps://towardsdatascience.com/install-custom-python-libraries-from-private-pypi-on-databricks-6a7669f6e6fd
Hello Databricks Community! We are getting so excited about the upcoming event of the year Data & AI Summit! If you still need to sign up, visit our registration page [link]! We have an in-person and virtual option for attending this year. In prepara...
Is there any known issues affecting the creation of clusters? I've been unable to get any clusters to start today so far!Have received this error "Cluster terminated.Reason:Unexpected launch failure"Help!
I am running a massive history of about 250gb ~6mil phone call transcriptions (json read in as raw text) from a raw -> bronze pipeline in Azure Databricks using pyspark. The source is mounted storage and is continuously having files added and we do n...
I have loaded the csv file from my local PC to data bricks. Now, when i am trying to query the table over column its throwing error. Note - select * from table_name is working but select column_name from table_name is throwing error.
Hey, So currently I'm using Pyspark and utilizing SparkListener to track metrics of my spark jobs. The problem I can't seem to solve is why in databricks the listener cannot be removed with removeSparkListener as it still is attached to the context.#...
Hi @CJ W​, Please go through this GitHub link which resolves this patch:-https://github.com/apache/spark/pull/16382/commits/347bb0e74e16cf19511ccfa3ed763db71c57aa4fPlease let us know if it helps you in any way.
Hello, we are developers who have been creating a system in Databricks with Scala. We enabled the Git feature, so the project is in a repository. The project has a lot of notebooks and a lot of calls to other notebooks. Sometimes it is a little overw...
it is true that we can't work without data bricks but we can develop an IDE and send the jar to databricks, this will allow us to create unit tests, and use the IDE capabilities (i.e fast navigation among classes).
Hey there @Ross Crill​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? It would be really helpful for the other members too.We'd love to hear fro...
Hey @Janga Reddy​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!
