Get Started Resources
Explore essential resources to kickstart your journey with Databricks. Access tutorials, guides, and...
0 Posts
Activity in Resources
- 1298 Views
- 1 replies
- 1 kudos
Six steps to improve your RAG application’s data foundation
This post is written by Ellen Hirt, Senior Specialist Solutions Engineer, and Pascal Vogel, Solutions Architect. Over the past year, Databricks has supported many teams with building Retrieval Augmented Generation (RAG) applications. We've noticed th...
- 1298 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
Great post! Do you have a recommended set of best practices for clustering queries when doing embedding model evaluation and selection? I've found that without clustering, users have to choose to either use a broad average on their whole dataset, or ...
- 1 kudos
- 9054 Views
- 3 replies
- 2 kudos
How To Setup Your First Federated Lakehouse
Introduction Announced at the Data + AI Summit in June 2023, Lakehouse Federation in Databricks is a groundbreaking new capability that allows you to query data across external data sources - including Snowflake, Synapse, many others and even Databri...
- 9054 Views
- 3 replies
- 2 kudos
Latest Reply
- 2 kudos
thanks @mido1978 for your reply. So if i understand correctly it's been looked at by the Databricks development team and that's a good news ! (for now yes we are already copying the delta tables out of OneLake to Gen2 which obviously defeat the purp...
- 2 kudos
- 87 Views
- 0 replies
- 0 kudos
End-to-End Structured Extraction with LLM – Part 2: Fine-Tuning with Synthetic Data
This is part 2 of a two-part series on Structured Extraction with LLM on Databricks. Read here for part 1! Introduction In part 1 of this series, I demonstrated how to use a large language model (LLM) with structured output and AI_QUERY to perform ...
- 87 Views
- 0 replies
- 0 kudos
- 117 Views
- 0 replies
- 0 kudos
The Future 50: The companies most likely to adapt, thrive, and grow
The global economy in 2024 is a tale of two forces: optimism sparked by falling interest rates, and the uncertainty caused by geopolitical unrest. As companies focus on sustainable growth, such a volatile environment inevitably tests their resilience...
- 117 Views
- 0 replies
- 0 kudos
- 139 Views
- 0 replies
- 0 kudos
Databricks Brings AI to the Enterprise using NVIDIA AI and Accelerated Computing
The world of artificial intelligence (AI) and data analytics is about to get a significant boost, thanks to Databricks’ collaboration with NVIDIA. This work brings together the cutting-edge capabilities of Databricks’ Mosaic AI platform and NVIDIA AI...
- 139 Views
- 0 replies
- 0 kudos
- 37 Views
- 0 replies
- 0 kudos
Understanding Databricks Workspace IP Access List
What is a Databricks Workspace IP Access List?The Databricks Workspace IP Access List is a security feature that allows administrators to control access to the Databricks workspace by specifying which IP addresses or IP ranges are allowed or denied a...
- 37 Views
- 0 replies
- 0 kudos
- 32669 Views
- 6 replies
- 6 kudos
How to efficiently ingest Dataverse Common Data Model (CDM) tables with Databricks
Organizations on Microsoft Azure commonly use Microsoft Dynamics 365 as their CRM or ERP application. Leveraging these applications' data for Business Intelligence, Machine Learning, or Artificial Intelligence gives organizations a real competitive ...
- 32669 Views
- 6 replies
- 6 kudos
Latest Reply
- 6 kudos
Apologies my above comment on the datetime format is wrong of course, the format in the article is correct, I just forgot to add the legacy timeParserPolicy settings. #To Parse Timestamps correctlyspark.conf.set("spark.sql.legacy.timeParserPolicy", "...
- 6 kudos
- 671 Views
- 2 replies
- 5 kudos
Democratize the data: Secure data exchange with Databricks Delta Sharing and Clean Rooms - Part I
Data sharing among collaborators has become increasingly important across industries, but it must be done in compliance with data protection regulations like CCPA and GDPR. Databricks offers two key solutions to enable secure and compliant data colla...
- 671 Views
- 2 replies
- 5 kudos
- 44 Views
- 1 replies
- 0 kudos
Managed Tables vs External Tables in Unity Catalog - 2023.11.03
Have you ever wondered what the difference between managed and external tables is? Olivia Zhang, a Solutions Architect at Databricks, goes over the differences and explains when to use what using an example in this quick video! Target Audience - Dat...
- 44 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Here is a basic difference between managed table and external table:Managed Tables Managed tables are fully controlled by Databricks, including both the data and metadata lifecycle.Data is stored in a Databricks-managed storage location configured b...
- 0 kudos
- 4800 Views
- 2 replies
- 5 kudos
Databricks observability using Grafana and Prometheus
Databricks Observability using Grafana and Prometheus As software systems scale, the amount of data they process and the work they do grows with them. This is not limited only to cloud-native services but also data pipelines that support them. When y...
- 4800 Views
- 2 replies
- 5 kudos
Latest Reply
- 5 kudos
Thank you @aleksandar . Can you provide the link for Grafana dashboard file
- 5 kudos
- 335 Views
- 0 replies
- 0 kudos
Get Started With SQL Analytics and BI on Databricks
Upskill on SQL analytics and BI with three short self-paced videos As organizations seek to democratize their data, there is an increasing demand to enable users to better understand and work with data. Learn how to better understand and analyze data...
- 335 Views
- 0 replies
- 0 kudos
- 22 Views
- 0 replies
- 0 kudos
Systems Tables - 10.27.2023
In this video, a Senior Specialist Solutions Architect at Databricks, goes over Observability on Databricks and how this can be achieved using Systems Tables. Observability is the ability to monitor and understand how an application is behaving. Thi...
- 22 Views
- 0 replies
- 0 kudos
- 4314 Views
- 1 replies
- 4 kudos
Private and Dedicated Connectivity Patterns for Databricks Serverless Using Private Link - Part 1
Looking to securely connect your private resources to Databricks Serverless? In our latest blog series, we explore how Private Link offers a seamless, secure connection between your cloud environments and Databricks' serverless offerings. Learn how t...
- 4314 Views
- 1 replies
- 4 kudos
Latest Reply
- 4 kudos
Private and dedicated connectivity patterns for Databricks Serverless using Private Link allow organizations to securely connect to Databricks services without exposing their traffic to the public internet. This enhances security, compliance, and net...
- 4 kudos
- 29 Views
- 0 replies
- 0 kudos
Lakehouse Federation - Querying data in other warehouses 2023.11.02
Data teams face many challenges to quickly access the right data primarily due to data fragmentation, time and cost involved in consolidating data, and difficulties in managing data governance across many systems.With Lakehouse Federation capabiliti...
- 29 Views
- 0 replies
- 0 kudos
- 693 Views
- 1 replies
- 0 kudos
From Hive to Thrive: How Delta Lake Liquid Clustering Transforms Data Efficiency
This blog talks about common ways in which Hive-style partitioning is used as a workaround for efficient data storage. Liquid Clustering improves partitioning and zorder techniques by simplifying data layout decisions while optimizing query performan...
- 693 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
you have mentioned target size in picture, what it refers. also does it resolve concurrentAppend Excecption
- 0 kudos
