Hello, Community members! I hope this message finds you well. I wanted to reach out and inquire about your experience with DAIS and how you are absorbing all the valuable information it provides. Additionally, I would love to hear what aspects of Gen...
Dear DBricks Team, Could you kindly look at the Back End Service issues? Thanks a Million.
Hello, My requirement is to created a materialized view (Live table) from a downstream table. The down stream table gets refreshed frequently and hence the materialized view needs to be recalculated. I don't want the pipe line to be in continuous mod...
Hi @Chengcheng I am fine with manual refresh also and have tried "Refresh Table <TableName>" command but it doesn't refreshes the table with new data.The Full refresh command "REFRESH TABLE <tableName> FULL; is giving below error inside notebook.REFR...
I thought this would be a good place to share your favorite talks from the Databricks Data+AI Summit 2023 Heck feel free to share your favorite talk from a different conference or previous summit Today's talk 'Delta-rs, Apache Arrow, Polars, WASM: Is...
I am having an issue where my permissions aren't being passed to workflows. I can use the same cluster to access files in an S3 repo but if I try to do spark-submit or a jar task on workflows I am unable to reach my buckets. I am trying to use Unity ...
Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
Hi Databricks Team,@Anonymous I have successfully completed databricks certification and received the certificate with a URL in which i can see my certificate and badge respectively for databricks data engineering exam and Apache spark developer exa...
I have 4 months of data and I partitioned it on Year and Month column, so my parquet partition looks like Data is present inside each monthly partition folder in parquet format.Then I loaded data for July and also modified some values in August. Afte...
Hi Team, Need some advice on setting up a single workspace or multiple workspaces.Please read below for the problem statement - There is an ADLS account says ADLS1 which has a root folder say root1This root1 folder has multiple sub foldersteam1team2...
Hi @Loki Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your que...
how to stream data to powerbi?
Hi @krausedata We haven't heard from you since the last response from @karthik_p​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others. Al...
I have attended data ai summit 2023 and it was cool.Although i cant find my discount 75% for any certification.Any thoughts how can i get my voucher?
I'm surprised this is not easily available, I checked this page for example: https://docs.databricks.com/release-notes/runtime/releases.html
@fijoy use %pip list in notebook and it should show libraries installed
I have a notebook (notebook A) that needs to run on a specific cluster.I now want to programmatically trigger notebook A from another notebook (notebook B), which also needs to run on a specific cluster (different from the cluster for notebook A).How...
@fijoy go with workflow job creation and create multiple task and assign different job clusters and trigger notebooks
In designing architecture for deploying machine learning models in production, what are the tradeoffs between a central Model Registry vs. per-Workspace Model Registries?
Anyone else using structured streaming at the summit?
