- 6216 Views
- 3 replies
- 1 kudos
How to refresh materialized view (Delta Live Table) without running a pipeline every time.
Hello, My requirement is to created a materialized view (Live table) from a downstream table. The down stream table gets refreshed frequently and hence the materialized view needs to be recalculated. I don't want the pipe line to be in continuous mod...
- 6216 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Chengcheng I am fine with manual refresh also and have tried "Refresh Table <TableName>" command but it doesn't refreshes the table with new data.The Full refresh command "REFRESH TABLE <tableName> FULL; is giving below error inside notebook.REFR...
- 1 kudos
- 1282 Views
- 0 replies
- 0 kudos
dbdemos - Build your Chat Bot with Dolly memory issue
I'm trying to run the Dolly demo on a nc6s_v3 128 GB single node cluster but get the following out of memory. Does anyone have any suggestions as to why this is/how to solve it? ThxOutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GP...
- 1282 Views
- 0 replies
- 0 kudos
- 6653 Views
- 5 replies
- 2 kudos
Resolved! Please share your favorite talks from the Databricks Data+AI Summit 2023 :)
I thought this would be a good place to share your favorite talks from the Databricks Data+AI Summit 2023 Heck feel free to share your favorite talk from a different conference or previous summit Today's talk 'Delta-rs, Apache Arrow, Polars, WASM: Is...
- 6653 Views
- 5 replies
- 2 kudos
- 1129 Views
- 0 replies
- 0 kudos
Workflow permissions
I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
- 1129 Views
- 0 replies
- 0 kudos
- 1250 Views
- 0 replies
- 0 kudos
Using Delta Live Tables Structured Streaming for small batches?
Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
- 1250 Views
- 0 replies
- 0 kudos
- 3696 Views
- 1 replies
- 2 kudos
Free Certification vouchers in Databricks Rewards Store
When will the Databricks Community rewards store gets upgraded with new 100% certification vouchers.
- 3696 Views
- 1 replies
- 2 kudos
- 2 kudos
@KVNARK same i am also looking for 100% certification voucher
- 2 kudos
- 1141 Views
- 0 replies
- 0 kudos
Certification Number required to register for external certificate in organization.
Hi Databricks Team,@Anonymous I have successfully completed databricks certification and received the certificate with a URL in which i can see my certificate and badge respectively for databricks data engineering exam and Apache spark developer exa...
- 1141 Views
- 0 replies
- 0 kudos
- 1098 Views
- 0 replies
- 0 kudos
Missing 13.x runtimes on dockerhub
When is Databricks going to publish the 13.x runtimes on Dockerhub?We are using custom containers, but we need to upgrade to Pandas >= 2.0,.. and for that we need to upgrade Spark to >=3.4, and for that we need to upgrade our custom containers to the...
- 1098 Views
- 0 replies
- 0 kudos
- 1794 Views
- 0 replies
- 0 kudos
Update a part of parquet partition is deleting existing data.
I have 4 months of data and I partitioned it on Year and Month column, so my parquet partition looks like Data is present inside each monthly partition folder in parquet format.Then I loaded data for July and also modified some values in August. Afte...
- 1794 Views
- 0 replies
- 0 kudos
- 7437 Views
- 6 replies
- 6 kudos
Resolved! Setting Up Databricks Workspace
Hi Team, Need some advice on setting up a single workspace or multiple workspaces.Please read below for the problem statement - There is an ADLS account says ADLS1 which has a root folder say root1This root1 folder has multiple sub foldersteam1team2...
- 7437 Views
- 6 replies
- 6 kudos
- 6 kudos
Hi @Loki Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your que...
- 6 kudos
- 3048 Views
- 2 replies
- 3 kudos
streaming data to powerbi
how to stream data to powerbi?
- 3048 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @krausedata We haven't heard from you since the last response from @karthik_p​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others. Al...
- 3 kudos
- 3121 Views
- 2 replies
- 1 kudos
Resolved! Unity catalog with fine grain access
When can I get fine grain access control in databricks
- 3121 Views
- 2 replies
- 1 kudos
- 1 kudos
I wanted this for some time and found that its implemented now. Thanks - https://learn.microsoft.com/en-us/azure/databricks/data-governance/table-acls/object-privileges
- 1 kudos
- 1380 Views
- 0 replies
- 0 kudos
Certification and data ai summit 2023
I have attended data ai summit 2023 and it was cool.Although i cant find my discount 75% for any certification.Any thoughts how can i get my voucher?
- 1380 Views
- 0 replies
- 0 kudos
- 7125 Views
- 4 replies
- 0 kudos
Where can I find a list of Databricks runtime versions and their corresponding Python versions?
I'm surprised this is not easily available, I checked this page for example: https://docs.databricks.com/release-notes/runtime/releases.html
- 7125 Views
- 4 replies
- 0 kudos
- 0 kudos
@fijoy use %pip list in notebook and it should show libraries installed
- 0 kudos
- 2326 Views
- 1 replies
- 0 kudos
How can I run one notebook from another notebook on a different cluster?
I have a notebook (notebook A) that needs to run on a specific cluster.I now want to programmatically trigger notebook A from another notebook (notebook B), which also needs to run on a specific cluster (different from the cluster for notebook A).How...
- 2326 Views
- 1 replies
- 0 kudos
- 0 kudos
@fijoy go with workflow job creation and create multiple task and assign different job clusters and trigger notebooks
- 0 kudos
