Learning more about the capabilities of unity catalog at the data and AI summit. I’m looking forward to implementing unity catalog in my current role. I’m hoping to better our documentation of the data lineage and improve data governance!
Just learned databricks warehouse as compared to snowflake. They actually have a very competitive product built in.will explore more.
We actually compared Snowflake and Databricks as part of a POC to determine what analytics architecture to deploy for the enterprise. Once Datsbricks released the SQL warehouse it was an obvious choice due to the open source framework and control ove...
So far I used only Hive, now I want to test Unity-CatalogI am following this guide:https://docs.databricks.com/data-governance/unity-catalog/get-started.htmlI see the table created under main catalog. Previsouly they were created under hive_metastore...
When data ai summit come to japan
or Europe, as it used to be
I am working on Generative AI, and would like to know more from databrick.
nice!what would you like to know from Databricks?
Even through we have option to use databricks cluster for SQL query , when we should use SQL warehouse ?
a sql warehouse can be used for interactive SQL querying.If you want to do batch processing using sql, a classic cluster is a better choice (because cheaper), but for interactive queries, performance is key. SQL warehouses are pretty fast and optimi...
What would be reason of teams choosing to have their data in #parquet instead of #delta.
no need for a log, always a full overwrite f.e.
Hello,What is the correct way to install packages from requierements.txt within databricks repo. Do I need to add some utils notebooks with additional scripts to my repo and run them before any of the script from the file? I suppose adding pip instal...
The .py files are very handy if you create classes etc. They contain modules which you can import into a notebook with the import statement. They are not meant to be run.
The summit is a great place to meet fellow data practitioners and learn more about how other companies are using Databricks so we can best leverage the full power of the tool. I am specially interested into hearing tips about better cost management t...
There was a session on migration to unity catalog today. Maybe you can check the videos. 
Hello everyone! My name is Rio Jia, currently pursuing my master's in data science at Vanderbilt University, and also have the privilege of interning in data science at Dell Technologies this summer. I learned a lot from the various sessions and expo...
The talk highlighted the benefits of using an open data lake for unified batch and streaming workloads and showcased features like Autoloader for data discovery, streaming triggers for seamless switching, and streaming aggregation for incremental com...
Had an amazing time at Data and AI summit 2023
It has been great so far! What has been your favorite session?I specially liked the talk from Appria around real time model monitoring.
My email id is bhatta.rohan@gmail.com. I am trying to reset my password but each time I am clicking on the link in the email and updating the password the page is going on loading. Please help.
I'm Kishore, Data Engineering Manager at Inari, where we use multiplex gene editing and our predictive design engine toolkit to design better yield and resilient seeds for crops such as Corn, Soy and Wheat. As a software engineer turned data engineer...
I work in ads management industry. Integrating marketplaces platforms, such as, Amazon, Google, Instacart etc. We are using databricks about 3 years. As we gathering several sources near-real time, Streaming and DLT are our main technologies nowadays...
