I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
When will the Databricks Community rewards store gets upgraded with new 100% certification vouchers.
@KVNARK same i am also looking for 100% certification voucher
Hi Databricks Team,@Anonymous I have successfully completed databricks certification and received the certificate with a URL in which i can see my certificate and badge respectively for databricks data engineering exam and Apache spark developer exa...
When is Databricks going to publish the 13.x runtimes on Dockerhub?We are using custom containers, but we need to upgrade to Pandas >= 2.0,.. and for that we need to upgrade Spark to >=3.4, and for that we need to upgrade our custom containers to the...
I have 4 months of data and I partitioned it on Year and Month column, so my parquet partition looks like Data is present inside each monthly partition folder in parquet format.Then I loaded data for July and also modified some values in August. Afte...
Hi Team, Need some advice on setting up a single workspace or multiple workspaces.Please read below for the problem statement - There is an ADLS account says ADLS1 which has a root folder say root1This root1 folder has multiple sub foldersteam1team2...
Hi @Loki Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your que...
how to stream data to powerbi?
Hi @krausedata We haven't heard from you since the last response from @karthik_p​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others. Al...
When can I get fine grain access control in databricks
I wanted this for some time and found that its implemented now. Thanks - https://learn.microsoft.com/en-us/azure/databricks/data-governance/table-acls/object-privileges
I have attended data ai summit 2023 and it was cool.Although i cant find my discount 75% for any certification.Any thoughts how can i get my voucher?
I'm surprised this is not easily available, I checked this page for example: https://docs.databricks.com/release-notes/runtime/releases.html
@fijoy use %pip list in notebook and it should show libraries installed
I have a notebook (notebook A) that needs to run on a specific cluster.I now want to programmatically trigger notebook A from another notebook (notebook B), which also needs to run on a specific cluster (different from the cluster for notebook A).How...
@fijoy go with workflow job creation and create multiple task and assign different job clusters and trigger notebooks
Yesterday I created a account in community but today i am unable to login i tried to rest but when i click or reset email link it goes in hang state. screenshot is attached.for Login i am using this url Login - Databricks Community Edition
adding @Anonymous and @Retired_mod for visibility to help with your request
What is the best way to explain to non-technical customers the difference between a data warehouse and a data lakehouse?
Hi @SigmaTSE ,Data warehouse is like an organized pantry with labeled jars of ingredients.Data lakehouse is like a magical fridge where you can store all kinds of ingredients without worrying about labels, allowing for more flexibility and creativity...
Hi Team,I have attended Data + AI Summit 2023! You event and have given and passed the databricks accredited lakehouse platform fundamentals certification.Could you please provide me the voucher. So that I could take another certification.Please find...
