So far I used only Hive, now I want to test Unity-CatalogI am following this guide:https://docs.databricks.com/data-governance/unity-catalog/get-started.htmlI see the table created under main catalog. Previsouly they were created under hive_metastore...
When data ai summit come to japan
or Europe, as it used to be
I am working on Generative AI, and would like to know more from databrick.
nice!what would you like to know from Databricks?
Even through we have option to use databricks cluster for SQL query , when we should use SQL warehouse ?
a sql warehouse can be used for interactive SQL querying.If you want to do batch processing using sql, a classic cluster is a better choice (because cheaper), but for interactive queries, performance is key. SQL warehouses are pretty fast and optimi...
What would be reason of teams choosing to have their data in #parquet instead of #delta.
no need for a log, always a full overwrite f.e.
Hello,What is the correct way to install packages from requierements.txt within databricks repo. Do I need to add some utils notebooks with additional scripts to my repo and run them before any of the script from the file? I suppose adding pip instal...
The .py files are very handy if you create classes etc. They contain modules which you can import into a notebook with the import statement. They are not meant to be run.
The summit is a great place to meet fellow data practitioners and learn more about how other companies are using Databricks so we can best leverage the full power of the tool. I am specially interested into hearing tips about better cost management t...
There was a session on migration to unity catalog today. Maybe you can check the videos. 
Hello everyone! My name is Rio Jia, currently pursuing my master's in data science at Vanderbilt University, and also have the privilege of interning in data science at Dell Technologies this summer. I learned a lot from the various sessions and expo...
The talk highlighted the benefits of using an open data lake for unified batch and streaming workloads and showcased features like Autoloader for data discovery, streaming triggers for seamless switching, and streaming aggregation for incremental com...
Had an amazing time at Data and AI summit 2023
It has been great so far! What has been your favorite session?I specially liked the talk from Appria around real time model monitoring.
My email id is bhatta.rohan@gmail.com. I am trying to reset my password but each time I am clicking on the link in the email and updating the password the page is going on loading. Please help.
I'm Kishore, Data Engineering Manager at Inari, where we use multiplex gene editing and our predictive design engine toolkit to design better yield and resilient seeds for crops such as Corn, Soy and Wheat. As a software engineer turned data engineer...
I work in ads management industry. Integrating marketplaces platforms, such as, Amazon, Google, Instacart etc. We are using databricks about 3 years. As we gathering several sources near-real time, Streaming and DLT are our main technologies nowadays...
I used to be a Technical Recruiter and used data to understand hiring process and how to target the right candidates for niche roles. My company started hiring for the data analytics role and I thought of this as an opportunity pave my way into the a...
Sasha - Congrats on building your skills and having the courage to make a big jump! Very inspiring!
We just implemented Data Catalog, but in terms of a data dictionary that intertwines with a business glossary, what are people using out there? I need to have something our non technical users see to understand what can be queried.
Hi ArtG,I haven't tried it myself, but Atlan's website looks interesting - data catalog and glossary not just for Databricks but also for Snowflake, Salesforce, and a whole lot more
