Hi Databricks community, I have a blob storage folder that will receive file drops, with 3 files in each distinct drop: ex. Files A1,B1,C1 is one drop, A2,B2,C2 is the next drop. The DLT pipeline I've got setup has a lot of joins and aggregations, cu...
Hi Databricks Team,@Anonymous I have successfully completed databricks certification and received the certificate with a URL in which i can see my certificate and badge respectively for databricks data engineering exam and Apache spark developer exa...
I have 4 months of data and I partitioned it on Year and Month column, so my parquet partition looks like Data is present inside each monthly partition folder in parquet format.Then I loaded data for July and also modified some values in August. Afte...
Hi Team, Need some advice on setting up a single workspace or multiple workspaces.Please read below for the problem statement - There is an ADLS account says ADLS1 which has a root folder say root1This root1 folder has multiple sub foldersteam1team2...
Hi @Loki Thank you for posting your question in our community! We are happy to assist you. To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your que...
how to stream data to powerbi?
Hi @krausedata We haven't heard from you since the last response from @karthik_p​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others. Al...
I have attended data ai summit 2023 and it was cool.Although i cant find my discount 75% for any certification.Any thoughts how can i get my voucher?
I'm surprised this is not easily available, I checked this page for example: https://docs.databricks.com/release-notes/runtime/releases.html
@fijoy use %pip list in notebook and it should show libraries installed
I have a notebook (notebook A) that needs to run on a specific cluster.I now want to programmatically trigger notebook A from another notebook (notebook B), which also needs to run on a specific cluster (different from the cluster for notebook A).How...
@fijoy go with workflow job creation and create multiple task and assign different job clusters and trigger notebooks
In designing architecture for deploying machine learning models in production, what are the tradeoffs between a central Model Registry vs. per-Workspace Model Registries?
Anyone else using structured streaming at the summit?
#databrickscommunity
nice to meet you guys!!
The Expo Booths are a treasure hunt, met many interesting companies doing wonderful things in Generative AI #Data+AISummit
Hi. I have started using Databricks for my ML summer project and currently working with AutoML to select the best model and hyper parameters. Its interesting how it reduces the effort of writing the entire set of code and helps us save a lot of time....
What was everyone’s favorite session @ DAIS 2023?
